Add models for Virtual Registry for containers registry and upstreams
What does this MR do and why?
Add models for registry and upstreams
Add models for registry, upstream, and their join table
EE: true
Schema
classDiagram
class Reg["VirtualRegistries::Containers::Registry"]
Reg : Fkey group_id
Reg : Pkey id
Reg : String name (max 255)
Reg : String description (max 1024)
class RegU["VirtualRegistries::Containers::RegistryUpstream"]
RegU: Fkey group_id
RegU: Fkey registry_id
RegU: Fkey upstream_id
RegU: smallint position (default 1. Between 1 and 5).
class U["VirtualRegistries::Containers::Upstream"]
U : Fkey group_id
U : Pkey id
U : String url (required)
U : smallint cache_validity_hours (default 24)
U : jsonb username (AR encrypted)
U : jsonb password (AR encrypted)
class CR["VirtualRegistries::Containers::Cache::Entry"]
CR : Fkey group_id
CR : Pkey id
CR : Fkey upstream_id
CR : Timestamp upstream_checked_at
CR : Integer size
CR : SmallInt status
CR : String relative_path
CR : String file
CR : String object_storage_key
CR : String upstream_etag
CR : String content_type
CR : file_md5 bytea
CR : file_sha1 bytea
Reg "1" --> "0..*" RegU
RegU "1" --> "1" U
U "1" --> "0..*" CR
Implementation Notes
- To reduce the MR size, the models are added here with no behavior and no specs.
- The model behavior code and specs are added in Add model behavior for Virtual Registry for con... (!199508 - merged)
-
VirtualRegistries::Containers::Cache::Entry
will be implemented in another follow-up MR: Draft: Add VirtualRegistries::Containers::Cache... (!198320 - closed)
References
Screenshots or screen recordings
NA
How to set up and validate locally
These are plain DB schema changes, and the tables are not yet used by code at this point, so there isn't much to test.
I believe
Database Review
Up migration
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81078
main: == 20250710124555 CreateVirtualRegistriesContainerRegistries: migrating =======
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- create_table(:virtual_registries_container_registries, {:if_not_exists=>true})
main: -> 0.0311s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_registries\nADD CONSTRAINT check_e721e79322\nCHECK ( char_length(name) <= 255 )\nNOT VALID;\n")
main: -> 0.0008s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0003s
main: -- execute("ALTER TABLE virtual_registries_container_registries VALIDATE CONSTRAINT check_e721e79322;")
main: -> 0.0009s
main: -- execute("RESET statement_timeout")
main: -> 0.0006s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_registries\nADD CONSTRAINT check_68bb0189e2\nCHECK ( char_length(description) <= 1024 )\nNOT VALID;\n")
main: -> 0.0007s
main: -- execute("ALTER TABLE virtual_registries_container_registries VALIDATE CONSTRAINT check_68bb0189e2;")
main: -> 0.0005s
main: == 20250710124555 CreateVirtualRegistriesContainerRegistries: migrated (0.0975s)
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81078
ci: == [advisory_lock_connection] object_id: 143620, pg_backend_pid: 81194
ci: == 20250710124555 CreateVirtualRegistriesContainerRegistries: migrating =======
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- create_table(:virtual_registries_container_registries, {:if_not_exists=>true})
ci: -> 0.0132s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_registries\nADD CONSTRAINT check_e721e79322\nCHECK ( char_length(name) <= 255 )\nNOT VALID;\n")
ci: -> 0.0010s
ci: -- execute("SET statement_timeout TO 0")
ci: -> 0.0004s
ci: -- execute("ALTER TABLE virtual_registries_container_registries VALIDATE CONSTRAINT check_e721e79322;")
ci: -> 0.0299s
ci: -- execute("RESET statement_timeout")
ci: -> 0.0005s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_registries\nADD CONSTRAINT check_68bb0189e2\nCHECK ( char_length(description) <= 1024 )\nNOT VALID;\n")
ci: -> 0.0007s
ci: -- execute("ALTER TABLE virtual_registries_container_registries VALIDATE CONSTRAINT check_68bb0189e2;")
ci: -> 0.0006s
I, [2025-07-28T22:05:34.938182 #81093] INFO -- : Database: 'ci', Table: 'virtual_registries_container_registries': Lock Writes
I, [2025-07-28T22:05:34.939072 #81093] INFO -- : {:method=>"with_lock_retries", :class=>"gitlab:db:lock_writes", :message=>"Lock timeout is set", :current_iteration=>1, :lock_timeout_in_ms=>100}
I, [2025-07-28T22:05:34.940042 #81093] INFO -- : {:method=>"with_lock_retries", :class=>"gitlab:db:lock_writes", :message=>"Migration finished", :current_iteration=>1, :lock_timeout_in_ms=>100}
ci: == 20250710124555 CreateVirtualRegistriesContainerRegistries: migrated (0.1414s)
ci: == [advisory_lock_connection] object_id: 143620, pg_backend_pid: 81194
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81274
main: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: migrating ========
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- create_table(:virtual_registries_container_upstreams, {:if_not_exists=>true})
main: -> 0.0093s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_f565894f21\nCHECK ( char_length(url) <= 255 )\nNOT VALID;\n")
main: -> 0.0008s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0003s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_f565894f21;")
main: -> 0.0005s
main: -- execute("RESET statement_timeout")
main: -> 0.0006s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_6aea67ba05\nCHECK ( char_length(name) <= 255 )\nNOT VALID;\n")
main: -> 0.0005s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_6aea67ba05;")
main: -> 0.0004s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_4b5467ff6f\nCHECK ( char_length(description) <= 1024 )\nNOT VALID;\n")
main: -> 0.0006s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_4b5467ff6f;")
main: -> 0.0005s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_5eb36d5a0e\nCHECK ( cache_validity_hours >= 0 )\nNOT VALID;\n")
main: -> 0.0011s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_5eb36d5a0e;")
main: -> 0.0006s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_df12da285f\nCHECK ( num_nonnulls(username, password) = 2 OR num_nulls(username, password) = 2 )\nNOT VALID;\n")
main: -> 0.0007s
main: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_df12da285f;")
main: -> 0.0005s
main: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: migrated (0.1733s)
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81274
ci: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81349
ci: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: migrating ========
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- create_table(:virtual_registries_container_upstreams, {:if_not_exists=>true})
ci: -> 0.0136s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_f565894f21\nCHECK ( char_length(url) <= 255 )\nNOT VALID;\n")
ci: -> 0.0012s
ci: -- execute("SET statement_timeout TO 0")
ci: -> 0.0007s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_f565894f21;")
ci: -> 0.0006s
ci: -- execute("RESET statement_timeout")
ci: -> 0.0009s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_6aea67ba05\nCHECK ( char_length(name) <= 255 )\nNOT VALID;\n")
ci: -> 0.0006s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_6aea67ba05;")
ci: -> 0.0005s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_4b5467ff6f\nCHECK ( char_length(description) <= 1024 )\nNOT VALID;\n")
ci: -> 0.0007s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_4b5467ff6f;")
ci: -> 0.0007s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_5eb36d5a0e\nCHECK ( cache_validity_hours >= 0 )\nNOT VALID;\n")
ci: -> 0.0006s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_5eb36d5a0e;")
ci: -> 0.0006s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams\nADD CONSTRAINT check_df12da285f\nCHECK ( num_nonnulls(username, password) = 2 OR num_nulls(username, password) = 2 )\nNOT VALID;\n")
ci: -> 0.0012s
ci: -- execute("ALTER TABLE virtual_registries_container_upstreams VALIDATE CONSTRAINT check_df12da285f;")
ci: -> 0.0010s
I, [2025-07-28T22:06:23.948082 #81288] INFO -- : Database: 'ci', Table: 'virtual_registries_container_upstreams': Lock Writes
I, [2025-07-28T22:06:23.949224 #81288] INFO -- : {:method=>"with_lock_retries", :class=>"gitlab:db:lock_writes", :message=>"Lock timeout is set", :current_iteration=>1, :lock_timeout_in_ms=>100}
I, [2025-07-28T22:06:23.949809 #81288] INFO -- : {:method=>"with_lock_retries", :class=>"gitlab:db:lock_writes", :message=>"Migration finished", :current_iteration=>1, :lock_timeout_in_ms=>100}
ci: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: migrated (0.1589s)
ci: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81349
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81436
main: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: migrating
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- create_table(:virtual_registries_container_registry_upstreams, {:if_not_exists=>true})
main: -> 0.0089s
main: -- add_index(:virtual_registries_container_registry_upstreams, [:upstream_id, :registry_id], {:unique=>true, :name=>"virtual_reg_cont_reg_upstreams_on_upstream_and_registry_ids"})
main: -> 0.0012s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- execute("ALTER TABLE virtual_registries_container_registry_upstreams\nADD CONSTRAINT check_918a83b32b\nCHECK ( 1 <= position AND position <= 5 )\nNOT VALID;\n")
main: -> 0.0007s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0007s
main: -- execute("ALTER TABLE virtual_registries_container_registry_upstreams VALIDATE CONSTRAINT check_918a83b32b;")
main: -> 0.0006s
main: -- execute("RESET statement_timeout")
main: -> 0.0003s
main: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: migrated (0.0782s)
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81436
ci: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81506
ci: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: migrating
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- create_table(:virtual_registries_container_registry_upstreams, {:if_not_exists=>true})
ci: -> 0.0094s
ci: -- add_index(:virtual_registries_container_registry_upstreams, [:upstream_id, :registry_id], {:unique=>true, :name=>"virtual_reg_cont_reg_upstreams_on_upstream_and_registry_ids"})
ci: -> 0.0010s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- execute("ALTER TABLE virtual_registries_container_registry_upstreams\nADD CONSTRAINT check_918a83b32b\nCHECK ( 1 <= position AND position <= 5 )\nNOT VALID;\n")
ci: -> 0.0007s
ci: -- execute("SET statement_timeout TO 0")
ci: -> 0.0003s
ci: -- execute("ALTER TABLE virtual_registries_container_registry_upstreams VALIDATE CONSTRAINT check_918a83b32b;")
ci: -> 0.0006s
ci: -- execute("RESET statement_timeout")
ci: -> 0.0003s
I, [2025-07-28T22:07:12.500489 #81451] INFO -- : Database: 'ci', Table: 'virtual_registries_container_registry_upstreams': Lock Writes
I, [2025-07-28T22:07:12.501153 #81451] INFO -- : {:method=>"with_lock_retries", :class=>"gitlab:db:lock_writes", :message=>"Lock timeout is set", :current_iteration=>1, :lock_timeout_in_ms=>100}
I, [2025-07-28T22:07:12.501546 #81451] INFO -- : {:method=>"with_lock_retries", :class=>"gitlab:db:lock_writes", :message=>"Migration finished", :current_iteration=>1, :lock_timeout_in_ms=>100}
ci: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: migrated (0.0983s)
ci: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81506
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81574
main: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: migrating
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main: -> 0.0207s
main: -- index_exists?(:virtual_registries_container_registry_upstreams, [:group_id], {:name=>"idx_vreg_container_reg_upst_on_group", :algorithm=>:concurrently})
main: -> 0.0020s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0003s
main: -- add_index(:virtual_registries_container_registry_upstreams, [:group_id], {:name=>"idx_vreg_container_reg_upst_on_group", :algorithm=>:concurrently})
main: -> 0.0015s
main: -- execute("RESET statement_timeout")
main: -> 0.0006s
main: -- execute(" ALTER TABLE virtual_registries_container_registry_upstreams\n ADD CONSTRAINT constraint_vreg_container_reg_upst_on_unique_reg_pos\n UNIQUE (registry_id, position) DEFERRABLE INITIALLY DEFERRED;\n")
main: -> 0.0017s
main: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: migrated (0.0416s)
main: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81574
ci: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81650
ci: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: migrating
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- view_exists?(:postgres_partitions)
ci: -> 0.0318s
ci: -- index_exists?(:virtual_registries_container_registry_upstreams, [:group_id], {:name=>"idx_vreg_container_reg_upst_on_group", :algorithm=>:concurrently})
ci: -> 0.0037s
ci: -- execute("SET statement_timeout TO 0")
ci: -> 0.0005s
ci: -- add_index(:virtual_registries_container_registry_upstreams, [:group_id], {:name=>"idx_vreg_container_reg_upst_on_group", :algorithm=>:concurrently})
ci: -> 0.0030s
ci: -- execute("RESET statement_timeout")
ci: -> 0.0006s
ci: -- execute(" ALTER TABLE virtual_registries_container_registry_upstreams\n ADD CONSTRAINT constraint_vreg_container_reg_upst_on_unique_reg_pos\n UNIQUE (registry_id, position) DEFERRABLE INITIALLY DEFERRED;\n")
ci: -> 0.0024s
ci: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: migrated (0.0656s)
ci: == [advisory_lock_connection] object_id: 143600, pg_backend_pid: 81650
Down migration
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39646
main: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: reverting
main: -- transaction_open?(nil)
main: -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main: -> 0.0218s
main: -- indexes(:virtual_registries_container_registry_upstreams)
main: -> 0.0015s
main: -- current_schema(nil)
main: -> 0.0001s
main: -- execute(" ALTER TABLE virtual_registries_container_registry_upstreams\n DROP CONSTRAINT IF EXISTS constraint_vreg_container_reg_upst_on_unique_reg_pos;\n")
main: -> 0.0006s
main: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: reverted (0.0802s)
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39646
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39731
ci: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: reverting
ci: -- transaction_open?(nil)
ci: -> 0.0000s
ci: -- view_exists?(:postgres_partitions)
ci: -> 0.0355s
ci: -- indexes(:virtual_registries_container_registry_upstreams)
ci: -> 0.0015s
ci: -- current_schema(nil)
ci: -> 0.0003s
ci: -- execute(" ALTER TABLE virtual_registries_container_registry_upstreams\n DROP CONSTRAINT IF EXISTS constraint_vreg_container_reg_upst_on_unique_reg_pos;\n")
ci: -> 0.0010s
ci: == 20250710124564 AddConstraintsToVirtualRegistriesContainerRegistryUpstreamsPosition: reverted (0.0597s)
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39731
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39814
main: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: reverting
main: -- drop_table(:virtual_registries_container_registry_upstreams, {:if_exists=>true})
main: -> 0.0048s
main: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: reverted (0.0101s)
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39814
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39888
ci: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: reverting
ci: -- drop_table(:virtual_registries_container_registry_upstreams, {:if_exists=>true})
ci: -> 0.0056s
ci: == 20250710124560 CreateVirtualRegistriesContainerRegistryUpstreams: reverted (0.0147s)
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39888
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39954
main: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: reverting ========
main: -- drop_table(:virtual_registries_container_upstreams, {:if_exists=>true})
main: -> 0.0066s
main: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: reverted (0.0121s)
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 39954
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 40015
ci: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: reverting ========
ci: -- drop_table(:virtual_registries_container_upstreams, {:if_exists=>true})
ci: -> 0.0052s
ci: == 20250710124557 CreateVirtualRegistriesContainerUpstreams: reverted (0.0143s)
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 40015
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 40086
main: == 20250710124555 CreateVirtualRegistriesContainerRegistries: reverting =======
main: -- drop_table(:virtual_registries_container_registries, {:if_exists=>true})
main: -> 0.0074s
main: == 20250710124555 CreateVirtualRegistriesContainerRegistries: reverted (0.0637s)
main: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 40086
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 40154
ci: == 20250710124555 CreateVirtualRegistriesContainerRegistries: reverting =======
ci: -- drop_table(:virtual_registries_container_registries, {:if_exists=>true})
ci: -> 0.0048s
ci: == 20250710124555 CreateVirtualRegistriesContainerRegistries: reverted (0.0137s)
ci: == [advisory_lock_connection] object_id: 143580, pg_backend_pid: 40154
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #548783 (closed)
Edited by Radamanthus Batnag