Skip to content

Migration for adding name field to GCP configuration for streaming

Hitesh Raghuvanshi requested to merge 416572-name-migration into master

What does this MR do and why?

As part of issue #416572 (closed), we need to add support for name attribute for GCP logging configuration which streams audit events to GCP. This MR is first among 3 MRs that needs to be deployed for the issue. This is quite similar to !123354 (merged) which was for adding name attribute to audit_events_instance_external_audit_event_destinations and audit_events_external_audit_event_destinations for similar purpose.

This MR is doing following:

  1. Adding a migration to add name column to audit_events_google_cloud_logging_configurations table.
  2. Adding migrations to add text_limit to name column and a unique index on namespace_id and name columns.
  3. Adding a new concern ExternallyCommonDestinationable for containing common code for all the external audit event destinations related models.
  4. Migrated some common validation code from ExternallyDestinationable to ExternallyCommonDestinationable.

Migrations output

RAILS_ENV=development bin/rails db:migrate:up:main VERSION=20230726104022
main: == [advisory_lock_connection] object_id: 223880, pg_backend_pid: 81413
main: == 20230726104022 AddNameToGoogleCloudLoggingConfiguration: migrating =========
main: -- add_column(:audit_events_google_cloud_logging_configurations, :name, :text)
main:    -> 0.0025s
main: == 20230726104022 AddNameToGoogleCloudLoggingConfiguration: migrated (0.0059s)

main: == [advisory_lock_connection] object_id: 223880, pg_backend_pid: 81413

===========================================================================================================================================================

RAILS_ENV=development bin/rails db:migrate:down:main VERSION=20230726104022
main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 81878
main: == 20230726104022 AddNameToGoogleCloudLoggingConfiguration: reverting =========
main: -- remove_column(:audit_events_google_cloud_logging_configurations, :name, :text)
main:    -> 0.0022s
main: == 20230726104022 AddNameToGoogleCloudLoggingConfiguration: reverted (0.0096s)

main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 81878

===========================================================================================================================================================							

RAILS_ENV=development bin/rails db:migrate:up:main VERSION=20230726104547
main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 82790
main: == 20230726104547 AddTextLimitToGoogleCloudLoggingConfigurationName: migrating
main: -- transaction_open?()
main:    -> 0.0000s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- execute("ALTER TABLE audit_events_google_cloud_logging_configurations\nADD CONSTRAINT check_cdf6883cd6\nCHECK ( char_length(name) <= 72 )\nNOT VALID;\n")
main:    -> 0.0030s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- execute("ALTER TABLE audit_events_google_cloud_logging_configurations VALIDATE CONSTRAINT check_cdf6883cd6;")
main:    -> 0.0010s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230726104547 AddTextLimitToGoogleCloudLoggingConfigurationName: migrated (0.1412s)

main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 82790

===========================================================================================================================================================

RAILS_ENV=development bin/rails db:migrate:down:main VERSION=20230726104547
main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 83241
main: == 20230726104547 AddTextLimitToGoogleCloudLoggingConfigurationName: reverting
main: -- transaction_open?()
main:    -> 0.0000s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- execute("            ALTER TABLE audit_events_google_cloud_logging_configurations\n            DROP CONSTRAINT IF EXISTS check_cdf6883cd6\n")
main:    -> 0.0009s
main: == 20230726104547 AddTextLimitToGoogleCloudLoggingConfigurationName: reverted (0.0262s)

main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 83241

===========================================================================================================================================================

RAILS_ENV=development bin/rails db:migrate:up:main VERSION=20230726104616
main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 84153
main: == 20230726104616 AddIndexToGoogleCloudLoggingConfiguration: migrating ========
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.1210s
main: -- index_exists?(:audit_events_google_cloud_logging_configurations, [:namespace_id, :name], {:unique=>true, :name=>"uniq_google_cloud_logging_configuration_namespace_id_and_name", :algorithm=>:concurrently})
main:    -> 0.0031s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0005s
main: -- add_index(:audit_events_google_cloud_logging_configurations, [:namespace_id, :name], {:unique=>true, :name=>"uniq_google_cloud_logging_configuration_namespace_id_and_name", :algorithm=>:concurrently})
main:    -> 0.0025s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: == 20230726104616 AddIndexToGoogleCloudLoggingConfiguration: migrated (0.1437s)

main: == [advisory_lock_connection] object_id: 223860, pg_backend_pid: 84153

===========================================================================================================================================================

RAILS_ENV=development bin/rails db:migrate:down:main VERSION=20230726104616
main: == [advisory_lock_connection] object_id: 223920, pg_backend_pid: 84592
main: == 20230726104616 AddIndexToGoogleCloudLoggingConfiguration: reverting ========
main: -- transaction_open?()
main:    -> 0.0001s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.1019s
main: -- indexes(:audit_events_google_cloud_logging_configurations)
main:    -> 0.0031s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0004s
main: -- remove_index(:audit_events_google_cloud_logging_configurations, {:algorithm=>:concurrently, :name=>"uniq_google_cloud_logging_configuration_namespace_id_and_name"})
main:    -> 0.0021s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230726104616 AddIndexToGoogleCloudLoggingConfiguration: reverted (0.1275s)

main: == [advisory_lock_connection] object_id: 223920, pg_backend_pid: 84592

How to set up and validate locally

  1. For testing this we can create a GCP config from GitLab dashboard for a group by following doc.
  2. Then in rails console, by running AuditEvents::GoogleCloudLoggingConfiguration.last.name we can check if it is giving some output like Destination_e4db7ba2-fb46-4abd-885d-c923f9b111c4 or not.
  3. Also, for providing name, a destination can be created manually in rails console by running AuditEvents::GoogleCloudLoggingConfiguration.create!(name: "GCPcon", group: Group.first, google_project_id_name: "somename2", client_email: "abcd@example.com", private_key: "asfregregjbjkhbjkdbvkjbbr")

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #416572 (closed)

Edited by Hitesh Raghuvanshi

Merge request reports