Adding migrations for gcp config name backfilling and non null column
What does this MR do and why?
As part of issue #416572 (closed), we need to add support for name attribute for GCP logging configuration which streams audit events to GCP.
This MR is backfilling data in recently added name
column in MR !127620 (merged) for existing entries in audit_events_google_cloud_logging_configurations
table.
This MR is quite similar to !124296 (merged) which was merged few days back for backfilling name column in audit_events_external_audit_event_destinations
table.
Notes around this migration
- There are currently only 7 entries on gitlab.com for
audit_events_google_cloud_logging_configurations
as of 4th August 2023. This feature was recently introduced. Because of less no. of entries the migration will not timeout. - Currently there is no way the
name
field inaudit_events_google_cloud_logging_configurations
table can be updated, so the values being backfilled will not impact any current functionality. - The default destination name being given by this backfill migration will be like
Destination <id>
, here id is the record id.
Migration logs
# up for backfilling name data
RAILS_ENV=development bin/rails db:migrate:up:main VERSION=20230804064817
main: == [advisory_lock_connection] object_id: 224600, pg_backend_pid: 88053
main: == 20230804064817 BackfillGoogleCloudLoggingName: migrating ===================
main: == 20230804064817 BackfillGoogleCloudLoggingName: migrated (0.0246s) ==========
main: == [advisory_lock_connection] object_id: 224600, pg_backend_pid: 88053
# down for backfilling name data
RAILS_ENV=development bin/rails db:migrate:down:main VERSION=20230804064817
main: == [advisory_lock_connection] object_id: 224600, pg_backend_pid: 87225
main: == 20230804064817 BackfillGoogleCloudLoggingName: reverting ===================
main: == 20230804064817 BackfillGoogleCloudLoggingName: reverted (0.0038s) ==========
main: == [advisory_lock_connection] object_id: 224600, pg_backend_pid: 87225
# up for setting name as non null
RAILS_ENV=development bin/rails db:migrate:up:main VERSION=20230804065052
main: == [advisory_lock_connection] object_id: 224600, pg_backend_pid: 89065
main: == 20230804065052 AddNotNullToGcpConfigName: migrating ========================
main: -- change_column_null(:audit_events_google_cloud_logging_configurations, :name, false)
main: -> 0.0024s
main: == 20230804065052 AddNotNullToGcpConfigName: migrated (0.0067s) ===============
# down for setting name as non null
RAILS_ENV=development bin/rails db:migrate:down:main VERSION=20230804065052
main: == [advisory_lock_connection] object_id: 224660, pg_backend_pid: 90094
main: == 20230804065052 AddNotNullToGcpConfigName: reverting ========================
main: -- change_column_null(:audit_events_google_cloud_logging_configurations, :name, true)
main: -> 0.0016s
main: == 20230804065052 AddNotNullToGcpConfigName: reverted (0.0060s) ===============
main: == [advisory_lock_connection] object_id: 224660, pg_backend_pid: 90094
Query plan
Query:
UPDATE "audit_events_google_cloud_logging_configurations" SET name = 'Destination ' || id
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/21086/commands/68889
How to set up and validate locally
- Run migration
rake db:migrate
- Verify every row in table
audit_events_google_cloud_logging_configurations
has a name value that is unique and follows patternDestination id
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #416572 (closed)
Edited by Hitesh Raghuvanshi