Fix audit event streaming to AWS S3 from new tables

Huzaifa Iftikharrequested to merge
567249_fix_streaming_aws_s3_events_from_new_tables into master

What does this MR do and why?

  1. Fix audit event streaming to AWS S3 from new tables
  2. The root cause is that the new tables (user_audit_events, project_audit_events, group_audit_events and instance_audit_events) don't have an entity_type column but we're trying to read audit_event['entity_type'] which is always nil.
  3. This MR updates the code to fetch the entity_type from payload instead of using audit_event.
  4. Closes #567249 (closed)

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. Ensure you have GitLab Ultimate license
  2. Enable the following feature flags
  3. stream_audit_events_from_new_tables
  4. audit_events_external_destination_streamer_consolidation_refactor
  5. Setup AWS S3 streaming destination following the instructions from this demo video.
  6. Make any actions that would generate an audit example, for example changing Merge request approval settings.
  7. Verify that the audit event is streamed to AWS S3.
  8. Also follow steps 4 and 5 after disabling audit_events_external_destination_streamer_consolidation_refactor, the streaming should still happen successfully.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports