Skip to content

Save taggings data to ci_runner_taggings

This is a ticket for the work of ci_runner_taggings

After #467196 (closed) and #467197 (closed) when a new job or runner is tagged, we must insert that data into their specific tables too along with the inserts into taggings.

This feature should be under a feature flag (disabled by default since we don't want self-managed to have this logic until we Finalize BackfillPartitionCiRunners migration for non-.com (#504277), and enabled on .com once we Finalize BackfillPartitionCiRunners migration on .com (#504276)).

Example for runners:

runner = Ci::Runner.last
runner.tag_list = ['ruby', 'docker']
runner.save!

This should trigger the following queries:

INSERT INTO "tags" ("name") VALUES ('ruby'), ('docker') ON CONFLICT ("name") DO NOTHING RETURNING "id";
INSERT INTO "taggings" ("tag_id", "taggable_type", "context", "created_at", "taggable_id") VALUES (5, 'Ci::Runner', 'tags', '2024-06-12 12:42:11.681466', 208740) RETURNING "id"; 
INSERT INTO "taggings" ("tag_id", "taggable_type", "context", "created_at", "taggable_id") VALUES (6, 'Ci::Runner', 'tags', '2024-06-12 12:42:11.684550', 208740) RETURNING "id";
INSERT INTO "ci_runner_taggings" ("tag_id", "runner_id", "runner_type", "sharding_key_id") VALUES (5, 208740, 3, 296), (6, 208740, 3, 296) RETURNING "id"; -- bulk insert by default

Similarly, reading runner.tag_list should generate this query when reading from taggings:

[11] pry(main)> runner.tag_list
  ActsAsTaggableOn::Tag Load (0.5ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 208740 AND "taggings"."taggable_type" = 'Ci::Runner' AND (taggings.context = 'tags' AND taggings.tagger_id IS NULL)
=> ["docker", "ruby"]

But tag_list should implement reading from the new table too under a feature flag and generate this query:

[11] pry(main)> job.tag_list
  ActsAsTaggableOn::Tag Load (0.5ms)  SELECT "tags".* FROM "tags" INNER JOIN "ci_runner_taggings" ON "tags"."id" = "ci_runner_taggings"."tag_id" WHERE "ci_runner_taggings"."runner_id" = 208740;
=> ["docker", "ruby"]

https://gitlab.com/gitlab-org/gitlab/-/blob/ebf1aa86b28adf1b91497a41f1ef3d9d8d4f36f7/app/models/concerns/ci/metadatable.rb#L60-117 implements a similar pattern from reading/writing the same data to multiple tables.

Concerns

  • We need to be sure that we can rely on either: 1. all records having been migrated to ci_runners_e59bb2812d, or 2. we copy the record from ci_runners before adding the tagging.
  • We should take #471398 into consideration by resolving duplicate tags when importing to the new table.