Skip to content

Create API for instance streaming headers

Hitesh Raghuvanshi requested to merge 404560-crud-streaming-headers into master

What does this MR do and why?

This is the second MR for issue #404560 (closed), where we need to add support for streaming headers for instance level external audit event destinations.

In the first MR !120997 (merged), migrations and models for table instance_audit_events_streaming_headers were added. This MR is adding GraphQL API for creation of the streaming headers for instance level external audit event destinations. APIs for listing, updation and deletion of headers will be added in subsequent MRs.

These headers will be sent in the request to the corresponding destinations. These are very similar to streaming headers for group level external audit event destinations. Upto 20 headers can be created per destination and no two headers for same destination can have same key.

GraphQL command

mutation {
  auditEventsStreamingInstanceHeadersCreate(input: 
    { 
      key: "header1",
      value: "header1value",
      destinationId: "<GID of corresponding InstanceExternalAuditEventDestination>"
    }) {
    errors
    header {
      id
    }
  }
}

SQL query plan

Note: Since the destinations work on instance level and no destination has been added to gitlab production instance, the rows being parsed are 0.

Find destination query

Query
SELECT 
  "audit_events_instance_external_audit_event_destinations".* 
FROM 
  "audit_events_instance_external_audit_event_destinations" 
WHERE 
  "audit_events_instance_external_audit_event_destinations"."id" = 3

Query plan: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/19185/commands/63316

Check if a header with same key exist for destination

Query
SELECT 
  1 AS one 
FROM 
  "instance_audit_events_streaming_headers" 
WHERE 
  "instance_audit_events_streaming_headers"."key" = 'foo' 
  AND "instance_audit_events_streaming_headers"."instance_external_audit_event_destination_id" = 3 
LIMIT 
  1

Query plan: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/19185/commands/63317

Insert header for destination

Query
INSERT INTO "instance_audit_events_streaming_headers" (
  "created_at", "updated_at", "instance_external_audit_event_destination_id", 
  "key", "value"
) 
VALUES 
  (
    '2023-06-05 12:56:27.752855', '2023-06-05 12:56:27.752855', 
    3, 'foo', 'bar'
  ) RETURNING "id"

Query plan: Since on the production gitlab instance, no instance level destination exists so we can't create a header for it.

How to set up and validate locally

  1. Enable the feature flag from rails console by running Feature.enable(:ff_external_audit_events).
  2. Open graphql-explorer console in browser by opening url http://127.0.0.1:3000/-/graphql-explorer.
  3. Run following mutation in the graphql explorer and note down the id, which would be something like gid://gitlab/AuditEvents::InstanceExternalAuditEventDestination/<id> from output.
mutation {
  instanceExternalAuditEventDestinationCreate(input: { destinationUrl: "https://www.example.com"}) {
    errors
    instanceExternalAuditEventDestination {
      destinationUrl
      id
    }
  }
}
  1. Run the following graphql command in explorer for creating a header created for destination created in step 3:
mutation {
  auditEventsStreamingInstanceHeadersCreate(input: 
    { 
      key: "header1",
      value: "header1value",
      destinationId: "gid://gitlab/AuditEvents::InstanceExternalAuditEventDestination/<id>"
    }) {
    errors
    header {
      id
      key
      value
    }
  }
}
  1. Verify on rails console that the header is created by running AuditEvents::Streaming::InstanceHeader.last.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #404560 (closed)

Edited by Hitesh Raghuvanshi

Merge request reports