Skip to content

Add verification header for streamed events

Max Woolf requested to merge 345424-audit-event-streaming-authenticity into master

What does this MR do and why?

  • Adds ability for audit stream consumers to verify origin of streamed events.
  • Adds new database column verification_token to ExternalAuditEventDestinations which is generated automatically by postgresql. The migration also adds tokens to existing destinations too.
  • Adds a new HTTP header to the streamed event which contains the verification token.
  • Adds GraphQL read-only support for the new field.

Database review

Migrate

== 20220105082217 AddVerificationTokenToExternalAeDestinations: migrating =====
-- add_column(:audit_events_external_audit_event_destinations, :verification_token, :text)
   -> 0.0050s
== 20220105082217 AddVerificationTokenToExternalAeDestinations: migrated (0.0051s)

== 20220119094023 AddUniqueIndexToAedVerificationToken: migrating =============
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:audit_events_external_audit_event_destinations, :verification_token, {:unique=>true, :name=>"index_audit_events_external_audit_on_verification_token", :algorithm=>:concurrently})
   -> 0.0051s
-- execute("SET statement_timeout TO 0")
   -> 0.0010s
-- add_index(:audit_events_external_audit_event_destinations, :verification_token, {:unique=>true, :name=>"index_audit_events_external_audit_on_verification_token", :algorithm=>:concurrently})
   -> 0.0096s
-- execute("RESET statement_timeout")
   -> 0.0007s
== 20220119094023 AddUniqueIndexToAedVerificationToken: migrated (0.0274s) ====

== 20220119094503 PopulateAuditEventStreamingVerificationToken: migrating =====
== 20220119094503 PopulateAuditEventStreamingVerificationToken: migrated (0.3187s)

Rollback

== 20220119094503 PopulateAuditEventStreamingVerificationToken: reverting =====
== 20220119094503 PopulateAuditEventStreamingVerificationToken: reverted (0.0000s)

== 20220119094023 AddUniqueIndexToAedVerificationToken: reverting =============
-- transaction_open?()
   -> 0.0001s
-- indexes(:audit_events_external_audit_event_destinations)
   -> 0.0110s
-- execute("SET statement_timeout TO 0")
   -> 0.0066s
-- remove_index(:audit_events_external_audit_event_destinations, {:algorithm=>:concurrently, :name=>"index_audit_events_external_audit_on_verification_token"})
   -> 0.0185s
-- execute("RESET statement_timeout")
   -> 0.0062s
== 20220119094023 AddUniqueIndexToAedVerificationToken: reverted (0.0517s) ====

== 20220117082611 AddTextLimitToExadVerificationTokens: reverting =============
-- transaction_open?()
   -> 0.0000s
-- transaction_open?()
   -> 0.0000s
-- execute("ALTER TABLE audit_events_external_audit_event_destinations\nDROP CONSTRAINT IF EXISTS check_8ec80a7d06\n")
   -> 0.0043s
== 20220117082611 AddTextLimitToExadVerificationTokens: reverted (0.0682s) ====

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

  • Create some external audit event streaming destinations on a group. (Docs) - (use something like RequestBin as a desintation)
  • Make sure the migration in this MR has been run locally.
    • Verify that the verification_token column has been created and populated (with randomly generated keys) for all existing desintations.
  • Trigger the creation of audit events within a group (create projects, add members, etc.)
  • Look at the streamed audit events, and ensure that they contain the X-Gitlab-Event-Streaming-Token header with the appropriate value.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #345424 (closed)

Edited by Max Woolf

Merge request reports