Add Geo pipeline ref created event to sync pipeline refs to secondary
What does this MR do and why?
This adds a Geo event for when we create a persistent pipeline ref, to replicate this creation to all Geo secondary sites, instead of waiting for the next repository sync.
📰 Context
With !81989 (merged), in %14.9, we've changed the refspec that runners use, from:
+#{pipeline.sha}:refs/pipelines/#{pipeline.id}
To:
+refs/pipelines/#{pipeline.id}:refs/pipelines/#{pipeline.id}
This resulted in runners pulling from a secondary not working anymore, as a secondary doesn't get the new ref when the pipeline is created, rather on the next sync (which may not happen for a while if the repo is not particularly active)
Related to #360816 (closed)
🐘 Migration outputs
$ scripts/db_tasks db:migrate
main: == 20220601130717 CreateGeoPipelineRefCreatedEvents: migrating ================
main: -- create_table(:geo_pipeline_ref_created_events, {:id=>:bigserial})
main: -> 0.0059s
main: -- add_column(:geo_event_log, :pipeline_ref_created_event_id, :integer, {:limit=>8})
main: -> 0.0016s
main: == 20220601130717 CreateGeoPipelineRefCreatedEvents: migrated (0.0088s) =======
main: == 20220601132057 AddIndexToGeoEventLogPipelineRefCreatedEventId: migrating ===
main: -- transaction_open?()
main: -> 0.0000s
main: -- index_exists?(:geo_event_log, :pipeline_ref_created_event_id, {:name=>"index_geo_event_log_on_pipeline_ref_created_event_id", :algorithm=>:concurrently})
main: -> 0.0161s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0007s
main: -- add_index(:geo_event_log, :pipeline_ref_created_event_id, {:name=>"index_geo_event_log_on_pipeline_ref_created_event_id", :algorithm=>:concurrently})
main: -> 0.0022s
main: -- execute("RESET statement_timeout")
main: -> 0.0007s
main: -- transaction_open?()
main: -> 0.0000s
main: -- foreign_keys(:geo_event_log)
main: -> 0.0054s
main: -- transaction_open?()
main: -> 0.0000s
main: -- execute("ALTER TABLE geo_event_log\nADD CONSTRAINT fk_4614ed001f\nFOREIGN KEY (pipeline_ref_created_event_id)\nREFERENCES geo_pipeline_ref_created_events (id)\nON DELETE CASCADE\nNOT VALID;\n")
main: -> 0.0015s
main: -- execute("ALTER TABLE geo_event_log VALIDATE CONSTRAINT fk_4614ed001f;")
main: -> 0.0017s
main: == 20220601132057 AddIndexToGeoEventLogPipelineRefCreatedEventId: migrated (0.0517s)
main: == 20220606015514 AddIndexToGeoPipelineRefCreatedEventsPipelineId: migrating ==
main: -- transaction_open?()
main: -> 0.0000s
main: -- index_exists?(:geo_pipeline_ref_created_events, :pipeline_id, {:name=>"index_geo_pipeline_ref_created_events_on_pipeline_id", :algorithm=>:concurrently})
main: -> 0.0024s
main: -- add_index(:geo_pipeline_ref_created_events, :pipeline_id, {:name=>"index_geo_pipeline_ref_created_events_on_pipeline_id", :algorithm=>:concurrently})
main: -> 0.0016s
main: == 20220606015514 AddIndexToGeoPipelineRefCreatedEventsPipelineId: migrated (0.0095s)
$ scripts/db_tasks db:migrate:down VERSION=20220606015514
main: == 20220606015514 AddIndexToGeoPipelineRefCreatedEventsPipelineId: reverting ==
main: -- transaction_open?()
main: -> 0.0000s
main: -- index_exists?(:geo_pipeline_ref_created_events, :pipeline_id, {:name=>"index_geo_pipeline_ref_created_events_on_pipeline_id", :algorithm=>:concurrently})
main: -> 0.0169s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0007s
main: -- remove_index(:geo_pipeline_ref_created_events, {:name=>"index_geo_pipeline_ref_created_events_on_pipeline_id", :algorithm=>:concurrently, :column=>:pipeline_id})
main: -> 0.0046s
main: -- execute("RESET statement_timeout")
main: -> 0.0007s
main: == 20220606015514 AddIndexToGeoPipelineRefCreatedEventsPipelineId: reverted (0.0357s)
$ scripts/db_tasks db:migrate:down VERSION=20220601132057
main: == 20220601132057 AddIndexToGeoEventLogPipelineRefCreatedEventId: reverting ===
main: -- transaction_open?()
main: -> 0.0000s
main: -- remove_foreign_key(:geo_event_log, {:column=>:pipeline_ref_created_event_id})
main: -> 0.0105s
main: -- transaction_open?()
main: -> 0.0000s
main: -- indexes(:geo_event_log)
main: -> 0.0117s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0005s
main: -- remove_index(:geo_event_log, {:algorithm=>:concurrently, :name=>"index_geo_event_log_on_pipeline_ref_created_event_id"})
main: -> 0.0013s
main: -- execute("RESET statement_timeout")
main: -> 0.0005s
main: == 20220601132057 AddIndexToGeoEventLogPipelineRefCreatedEventId: reverted (0.0452s)
$ scripts/db_tasks db:migrate:down VERSION=20220601130717
main: == 20220601130717 CreateGeoPipelineRefCreatedEvents: reverting ================
main: -- remove_column(:geo_event_log, :pipeline_ref_created_event_id, :integer, {:limit=>8})
main: -> 0.0032s
main: -- drop_table(:geo_pipeline_ref_created_events, {:id=>:bigserial})
main: -> 0.0021s
main: == 20220601130717 CreateGeoPipelineRefCreatedEvents: reverted (0.0101s) =======
How to set up and validate locally
It's probably easiest to run the ./qa/specs/features/browser_ui/6_release/pages/pages_pipeline_spec.rb:34
test - it would fail before this branch, it should succeed on this branch. (I've personally tested this on a GET env)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.