Add Switch module for Ci::BuildMetadata

changed milestone to %15.4

added CI data partitioning FY23Q3 OKR-FY24Q2 VerifyP2 backend ciscaling devopsverify grouppipeline execution sectionops typemaintenance workflowin dev + 1 deleted label

changed milestone to %15.6

added workflowready for development label and removed workflowin dev label

cc @grzesiek @mbobin

changed the description

mentioned in epic &7522

added to epic &5417

Thanks for creating the issue @morefice! I agree that we need to work on it!

I think that we should just reuse @mbobin's PoC, with using Ci::BuildsMetadata::Partitioned. In there we should be able to use feature flags to switch the behavior.

@grzesiek @mbobin

Looks like we need to focus on #377521 (closed) first.

Or we would need to implement this .partitioned scope to make sure all queries include the partition key.

@morefice I talked with @mbobin today about that in our coffee chat, and it seems that we just need to figure it out how to switch to reading from a routing table during the runtime (preferably using feature flags).

One way to do that is to use lambda associations, like has_one :something, ->() { if scope else another scope }. In such case we would need to probably implement #377521 (closed) first, but there are other ways. Since Ci::BuildMetadata::Partitioned would presumably inherit from Ci::BuildMetadata, instances of these classes could be use interchangeably. Perhaps we can make it work without lambda associations 🤔 ?

@grzesiek @morefice I've started to poke at ActiveRecord and managed to switch tables when interacting directly with the model(it doesn't work for associations, yet 😼):

[10] pry(main)> Ci::BuildMetadata.last;nil
  Ci::BuildMetadata Load (0.2ms)  SELECT "ci_builds_metadata".* FROM "ci_builds_metadata" ORDER BY "ci_builds_metadata"."id" DESC LIMIT 1 /*application:console,db_config_name:ci,line:(pry):10:in `__pry__'*/
=> nil
[11] pry(main)> Feature.enable(:ci_partitioned_reads)
=> true
[12] pry(main)> Ci::BuildMetadata.last;nil
  Ci::BuildMetadata Load (0.3ms)  SELECT "p_ci_builds_metadata".* FROM "p_ci_builds_metadata" ORDER BY "p_ci_builds_metadata"."id" DESC LIMIT 1 /*application:console,db_config_name:ci,line:(pry):12:in `__pry__'*/
=> nil
[13] pry(main)> Feature.disable(:ci_partitioned_reads)
=> true
[14] pry(main)> Ci::BuildMetadata.last;nil
  Ci::BuildMetadata Load (0.3ms)  SELECT "ci_builds_metadata".* FROM "ci_builds_metadata" ORDER BY "ci_builds_metadata"."id" DESC LIMIT 1 /*application:console,db_config_name:ci,line:(pry):14:in `__pry__'*/
=> nil