-
@stanhu it would probably be safer if we also patch the
Ci::Runner
records, in addition toCi::PendingBuild
. Would you mind adding that for users that might come across this script? -
@pedropombeiro Which field in
Ci::Runner
records needs patching? Is that handled by theTagging
updates? -
@stanhu oh, you're right! It's been a while since I've looked at tags, and I assumed that
Ci::PendingBuild
andCi::Runner
worked the same way, should have checked it👍 -
@gerardo reported that fetching the tags might result in a statement timeout. We might want to disable statement timeouts there.
-
A came across this script linked from here: https://docs.gitlab.com/runner/faq/#no-unique-index-found-for-name
After successfully running it, should I manually (or should the ruby script) clean up the duplicate tag ID's in the 'tags' table once there are no references to them in the 'taggings' table?
-
Got the following error:
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/postgresql/database_statements.rb:19:in `exec': PG::UndefinedTable: ERROR: relation "dast_scanner_profiles_tags" does not exist (ActiveRecord::StatementInvalid) LINE 9: WHERE a.attrelid = '"dast_scanner_profiles_tags"'::regclass
Existing dast tables:
public | dast_pre_scan_verification_steps | table | gitlab public | dast_pre_scan_verifications | table | gitlab public | dast_profile_schedules | table | gitlab public | dast_profiles | table | gitlab public | dast_profiles_pipelines | table | gitlab public | dast_profiles_tags | table | gitlab public | dast_scanner_profiles | table | gitlab public | dast_scanner_profiles_builds | table | gitlab public | dast_site_profile_secret_variables | table | gitlab public | dast_site_profiles | table | gitlab public | dast_site_profiles_builds | table | gitlab public | dast_site_tokens | table | gitlab public | dast_site_validations | table | gitlab public | dast_sites | table | gitlab
Seems that tables have been dropped: #442744 (comment 1909999726)
And I'm confused between:
- FAQ
DROP INDEX IF EXISTS index_tags_on_name; -- Drop a potentially invalid index CREATE UNIQUE INDEX index_tags_on_name ON tags USING btree (name);
- and the previous comment:
REINDEX INDEX CONCURRENTLY index_tags_on_name; REINDEX INDEX CONCURRENTLY index_tags_on_name_trigram;
Edited by Marc Finet (Rtone) -
I'm was seeing the same error noted by @marcfinet_rtone above:
ERROR: relation "dast_scanner_profiles_tags" does not exist
. As also noted by @stanhu it doesn't actually resolve the duplicate tag values on it's own when it's running without error -- requiring users to also go into the pg console and run arbitrary SQL commands.Below is my attempt at an updated version of @stanhu 's original script that removes the breaking reference to the dropped table, adds on deletion of the duplicate records from the
tags
table, and also adds dropping and re-creating the tag name index. (In other words, it tries to be an single fix.). You can create the file locally and then run it in the gitlab-rails context with:sudo gitlab-rails runner /full/path/to/dedudpe.rb
Also created as it's own snippet at $4780138.
# enable query logging ActiveRecord::Base.logger = Logger.new(STDOUT) # query for the distinct set of "good" tags and any duplicates good_tag_ids = ActiveRecord::Base.connection.select_all('SELECT name, MIN(id) AS id FROM tags GROUP BY name').rows.to_h bad_tag_map = ActiveRecord::Base.connection.select_all('SELECT id, name FROM tags WHERE id NOT IN (SELECT MIN(id) FROM tags GROUP BY name)').rows.to_h bad_tags = bad_tag_map.keys tag_remap = bad_tag_map.map { |id, name| [id, good_tag_ids[name]] }.to_h # taggings table interface class TempTaggings < ActiveRecord::Base include EachBatch self.table_name = "taggings" end # tags table interface class TempTags < ActiveRecord::Base include EachBatch self.table_name = "tags" end # for pending builds with the duplicate tag id values, update them with the selected good tag id Ci::PendingBuild.all.each_batch do |batch| batch.each do |build| tag_ids = build.tag_ids next unless tag_ids & bad_tags build.tag_ids = tag_ids.map { |x| tag_remap.fetch(x, x) } build.save! end end # for taggings entries with the duplicate tag id values, update them with the selected good tag id TempTaggings.all.where(tag_id: bad_tags).each_batch(of: 10000) do |batch| batch.each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end # for additional gitlab ee specific tables, update any duplicate tag ids ith the selected good tag id if Gitlab.ee? models = [Dast::ProfileTag] models.each do |model| model.all.where(tag_id: bad_tags).each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end end # references to the duplicate tags should now be replaced, so we can remove the duplicate tag entries to allow for unique index TempTags.all.where(id: bad_tag_map.keys).each_batch(of: 10000) do |batch| batch.destroy_all end # drop the potentially bad index and re-create it TempTags.connection.exec_query("DROP INDEX IF EXISTS index_tags_on_name;") TempTags.connection.exec_query("CREATE UNIQUE INDEX index_tags_on_name ON tags USING btree (name);")
-
We're seeing a similar issue (and were sent here by GitLab support) when upgrading from 17.5.3 to 17.5.4, but unfortunately neither the original script, nor Kevin's fixed version (for the missing
dast_scanner_profile_tags
table issue) work in our case.Just in case anybody else is in a similar situation and doesn't find anything with this specific issue, here is the error message:
PG::ForeignKeyViolation: ERROR: insert or update on table "ci_build_tags_100" violates foreign key constraint "fk_rails_8284d35c66" DETAIL: Key (tag_id)=(95) is not present in table "tags".
-
@jangrewe Seems like the script would need to handle that table as well, maybe the following would work?
ActiveRecord::Base.logger = Logger.new(STDOUT) good_tag_ids = ActiveRecord::Base.connection.select_all('SELECT name, MIN(id) AS id FROM tags GROUP BY name').rows.to_h bad_tag_map = ActiveRecord::Base.connection.select_all('SELECT id, name FROM tags WHERE id NOT IN (SELECT MIN(id) FROM tags GROUP BY name)').rows.to_h bad_tags = bad_tag_map.keys tag_remap = bad_tag_map.map { |id, name| [id, good_tag_ids[name]] }.to_h class Tagging < ActiveRecord::Base include EachBatch self.table_name = "taggings" end class DastScannerProfilesTags < ActiveRecord::Base self.table_name = "dast_scanner_profiles_tags" end Ci::BuildTag.include EachBatch; Ci::BuildTag.all.each_batch do |batch| batch.each do |build_tag| tag_id = build_tag.tag_id next unless bad_tags.include?(tag_id) build_tag.update!(tag_id: tag_remap.fetch(tag_id, tag_id)) end end Ci::PendingBuild.all.each_batch do |batch| batch.each do |build| tag_ids = build.tag_ids next unless tag_ids & bad_tags build.tag_ids = tag_ids.map { |x| tag_remap.fetch(x, x) } build.save! end end Tagging.all.where(tag_id: bad_tags).each_batch(of: 10000) do |batch| batch.each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end if Gitlab.ee? models = [Dast::ProfileTag, DastScannerProfilesTags] models.each do |model| model.all.where(tag_id: bad_tags).each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end end
Edited by Pedro Pombeiro -
Thanks @pedropombeiro, as this is the non-working script (we don't have
dast_scanner_profile_tags
anymore), i assume the relevant lines are this?Ci::BuildTag.include EachBatch; Ci::BuildTag.all.each_batch do |batch| batch.each do |build_tag| tag_id = build_tag.tag_id next unless bad_tags.include?(tag_id) build_tag.update!(tag_id: tag_remap.fetch(tag_id, tag_id)) end end
Then i'll try to create a mashup of this and Kevin's script, to also handle the SQL.
😃 Edit: Would this work?
ActiveRecord::Base.logger = Logger.new(STDOUT) good_tag_ids = ActiveRecord::Base.connection.select_all('SELECT name, MIN(id) AS id FROM tags GROUP BY name').rows.to_h bad_tag_map = ActiveRecord::Base.connection.select_all('SELECT id, name FROM tags WHERE id NOT IN (SELECT MIN(id) FROM tags GROUP BY name)').rows.to_h bad_tags = bad_tag_map.keys tag_remap = bad_tag_map.map { |id, name| [id, good_tag_ids[name]] }.to_h class TempTaggings < ActiveRecord::Base include EachBatch self.table_name = "taggings" end class TempTags < ActiveRecord::Base include EachBatch self.table_name = "tags" end Ci::BuildTag.include EachBatch; Ci::BuildTag.all.each_batch do |batch| batch.each do |build_tag| tag_id = build_tag.tag_id next unless bad_tags.include?(tag_id) build_tag.update!(tag_id: tag_remap.fetch(tag_id, tag_id)) end end Ci::PendingBuild.all.each_batch do |batch| batch.each do |build| tag_ids = build.tag_ids next unless tag_ids & bad_tags build.tag_ids = tag_ids.map { |x| tag_remap.fetch(x, x) } build.save! end end TempTaggings.all.where(tag_id: bad_tags).each_batch(of: 10000) do |batch| batch.each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end if Gitlab.ee? models = [Dast::ProfileTag] models.each do |model| model.all.where(tag_id: bad_tags).each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end end TempTags.all.where(id: bad_tag_map.keys).each_batch(of: 10000) do |batch| batch.destroy_all end TempTags.connection.exec_query("DROP INDEX IF EXISTS index_tags_on_name;") TempTags.connection.exec_query("CREATE UNIQUE INDEX index_tags_on_name ON tags USING btree (name);")
Edited by Jan Grewe -
@jangrewe actually
Ci::BuildTag
is very similar toTempTaggings
(it should be namedCi::BuildTagging
tbh), so let's harmonize it:ActiveRecord::Base.logger = Logger.new(STDOUT) good_tag_ids = ActiveRecord::Base.connection.select_all('SELECT name, MIN(id) AS id FROM tags GROUP BY name').rows.to_h bad_tag_map = ActiveRecord::Base.connection.select_all('SELECT id, name FROM tags WHERE id NOT IN (SELECT MIN(id) FROM tags GROUP BY name)').rows.to_h bad_tags = bad_tag_map.keys tag_remap = bad_tag_map.map { |id, name| [id, good_tag_ids[name]] }.to_h class TempRunnerTagging < Ci::ApplicationRecord self.table_name = :ci_runner_taggings self.primary_key = :id end class TempTaggings < ActiveRecord::Base self.table_name = "taggings" end class TempTags < ActiveRecord::Base include EachBatch self.table_name = "tags" end Ci::PendingBuild.all.each_batch do |batch| batch.each do |build| tag_ids = build.tag_ids next unless tag_ids & bad_tags build.tag_ids = tag_ids.map { |x| tag_remap.fetch(x, x) } build.save! end end tagging_models = [Ci::BuildTag, TempRunnerTagging, TempTaggings] tagging_models .each do |tagging_model| tagging_model.include EachBatch; tagging_model.all.where(tag_id: bad_tags).each_batch(of: 10000) do |batch| batch.each do |row| new_tag_id = tag_remap.fetch(row.tag_id) if tagging_model == TempRunnerTagging row_exists = tagging_model.where(tag_id: new_tag_id, runner_id: row.runner_id).exists? if row_exists row.destroy! else row.update!(tag_id: new_tag_id) end else row.update!(tag_id: new_tag_id) end end end end if Gitlab.ee? tagging_models = [Dast::ProfileTag] tagging_models.each do |tagging_model| tagging_model.all.where(tag_id: bad_tags).each do |row| new_tag_id = tag_remap.fetch(row.tag_id) row.update!(tag_id: new_tag_id) end end end TempTags.all.where(id: bad_tag_map.keys).each_batch(of: 10000) do |batch| batch.destroy_all end TempTags.connection.exec_query("DROP INDEX IF EXISTS index_tags_on_name;") TempTags.connection.exec_query("CREATE UNIQUE INDEX index_tags_on_name ON tags USING btree (name);")
Edited by Pedro Pombeiro -
@pedropombeiro It sounds like we should move this script into a Rake task so we can properly test and manage it based on GitLab versions?
-
Thanks @pedropombeiro, i've used your harmonized version and was able to upgrade to 17.5.4 successfully, now upgrading to 17.7.0.
-
Please note that the
taggings
table is being replaced byci_runner_taggings
in %17.9, so the script above would need to be adapted accordingly. -
Created an issue to make this script a rake task: #518698 (closed)
-
The script will be available as
gitlab-rake gitlab:db:deduplicate_tags
once !181280 (merged) has finished reviews and is merged. -
Note:
taggings
table has been replaced by specialized tables (p_ci_build_tags
,ci_runner_taggings
) from %17.9.Reference: !179384 (merged) and !179404 (merged)
-
This script has been updated with $3700665 (comment 2284396203). We also found duplicate
p_ci_build_tags
entries that were somehow not prevented with the unique index. The script has been updated to remove duplicates before trying to remap values. -
-
Hello,
I'm facing issue running the rub script (in CE-v17.8.7) as well as the rake task after upgrading to version CE-v17.10.5. It's the same error in both the scenarios.
Output of rake task in CE-v17.10.5
[rocky@gitlab-server-22 ~]$ sudo gitlab-rake gitlab:db:deduplicate_tags I, [2025-04-30T07:06:43.450588 #8768] INFO -- : Deduplicating 7 tags for ci database I, [2025-04-30T07:06:43.547766 #8768] INFO -- : Updated tag_ids on a batch of 52 ci_pending_builds records I, [2025-04-30T07:06:44.572075 #8768] INFO -- : Updated tag_id 659 on p_ci_build_tags records to 658 rake aborted! ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_build_tags_100_tag_id_build_id_partition_id_idx" DETAIL: Key (tag_id, build_id, partition_id)=(406, 3700360, 100) already exists. /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:61:in `block (2 levels) in <class:ConnectionProxy>' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:88:in `block (3 levels) in deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:102:in `block (2 levels) in each_batch' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:102:in `block in each_batch' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:72:in `step' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:72:in `each_batch' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:87:in `block (2 levels) in deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:86:in `each' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:86:in `block in deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:83:in `each' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:83:in `deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:62:in `deduplicate_ci_tags' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:36:in `execute' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db/deduplicate_tags.rake:10:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:25:in `load' /opt/gitlab/embedded/bin/bundle:25:in `<main>' Caused by: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_build_tags_100_tag_id_build_id_partition_id_idx" DETAIL: Key (tag_id, build_id, partition_id)=(406, 3700360, 100) already exists. /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:61:in `block (2 levels) in <class:ConnectionProxy>' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:88:in `block (3 levels) in deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:102:in `block (2 levels) in each_batch' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:102:in `block in each_batch' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:72:in `step' /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:72:in `each_batch' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:87:in `block (2 levels) in deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:86:in `each' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:86:in `block in deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:83:in `each' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:83:in `deduplicate_ci_taggings' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:62:in `deduplicate_ci_tags' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/deduplicate_ci_tags.rb:36:in `execute' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db/deduplicate_tags.rake:10:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:25:in `load' /opt/gitlab/embedded/bin/bundle:25:in `<main>' Tasks: TOP => gitlab:db:deduplicate_tags (See full trace by running task with --trace)
Since this ruby task was failing, I proceeded with deleting the duplicate tags as mentioned below
DELETE FROM tags WHERE id NOT IN ( SELECT MIN(id) FROM tags GROUP BY name ); DROP INDEX IF EXISTS index_tags_on_name; CREATE UNIQUE INDEX index_tags_on_name ON tags USING btree (name); REINDEX INDEX CONCURRENTLY index_tags_on_name_trigram;
With this method, I'm not facing the 500 error code issue when creating the runners, but I lose the job logs of all those impacted pipelines where the tag (deleted) was used.PS: GitLab (CE) is selfhosted using omnibus installation on Rocky Linux 9
Could someone please help me with this issue?
Thanks in advance.
Edited by Pedro Pombeiro -
@cspencer1 I think you followed similar steps like @sreeram.s as you mentioned at omnibus-gitlab#8579 (comment 1947844123), could you please tell if you loose CI job trace, after performing the mentioned steps ?
Edited by Rajesh Jena
-
-
@stanhu After using your below delete query, we are loosing the CI job trace as mentioned by @sreeram.s in above comment :
DELETE FROM tags WHERE id NOT IN ( SELECT MIN(id) FROM tags GROUP BY name ); REINDEX INDEX CONCURRENTLY index_tags_on_name; REINDEX INDEX CONCURRENTLY index_tags_on_name_trigram;
Could you please suggest how to regain job trace?
-
Do NOT run
DELETE FROM tags WHERE id NOT IN ( SELECT MIN(id) FROM tags GROUP BY name )
unless you have properly de-duped the database with the script above.It sounds to me the CI builds table tags were not remapped properly, but I'm not sure why the job logs aren't showing.
Please file a ticket with support if you are a customer.
-
-
@stanhu We are using gitlab CE, I think we don't have possibility of customer support ?
While running the script ($3700665 (comment 2284396203)) to fix de-dup database, we face below error:
Error Message
[2025-05-05T06:26:28.201272 #3226] DEBUG -- : TRANSACTION (0.2ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ ROLLBACK /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params': PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_build_tags_100_tag_id_build_id_partition_id_idx" (ActiveRecord::RecordNotUnique) DETAIL: Key (tag_id, build_id, partition_id)=(314, 3698608, 100) already exists.from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:767:in `block in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:752:in `block in log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:743:in `log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:766:in `exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:745:in `execute_and_clear' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/marginalia-1.11.1/lib/marginalia.rb:91:in `execute_and_clear_with_marginalia' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:67:in `exec_delete' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract/database_statements.rb:175:in `update' ... 90 levels...
/opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params': ERROR: duplicate key value violates unique constraint "ci_build_tags_100_tag_id_build_id_partition_id_idx" (PG::UniqueViolation) DETAIL: Key (tag_id, build_id, partition_id)=(314, 3698608, 100) already exists.
from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:767:in `block in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:752:in `block in log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:743:in `log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:766:in `exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:745:in `execute_and_clear' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/marginalia-1.11.1/lib/marginalia.rb:91:in `execute_and_clear_with_marginalia' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:67:in `exec_delete' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract/database_statements.rb:175:in `update' ... 90 levels...
Could you please suggest.
-
That's not the right script. The script at the top (this snippet) is the updated one and covers that case (#523146 (closed)).
Edited by Stan Hu -
Thanks for prompt response @stanhu, Tried running below script
Script Used
ActiveRecord::Base.logger = Logger.new(STDOUT) good_tag_ids = ActiveRecord::Base.connection.select_all('SELECT name, MIN(id) AS id FROM tags GROUP BY name').rows.to_h bad_tag_map = ActiveRecord::Base.connection.select_all('SELECT id, name FROM tags WHERE id NOT IN (SELECT MIN(id) FROM tags GROUP BY name)').rows.to_h bad_tags = bad_tag_map.keys tag_remap = bad_tag_map.map { |id, name| [id, good_tag_ids[name]] }.to_h puts tag_remap class TempRunnerTagging < Ci::ApplicationRecord self.table_name = :ci_runner_taggings self.primary_key = :id end class TempTaggings < ActiveRecord::Base self.table_name = :taggings self.primary_key = :id end class TempTags < ActiveRecord::Base include EachBatch self.table_name = :tags self.primary_key = :id end class CiBuildTag < ActiveRecord::Base self.table_name = :p_ci_build_tags self.primary_key = :id end # Identify duplicate entries in p_ci_build_tags duplicates_query = <<-SQL SELECT tag_id, build_id, partition_id, COUNT(*), ARRAY_AGG(id) as duplicate_ids FROM public.p_ci_build_tags GROUP BY tag_id, build_id, partition_id HAVING COUNT(*) > 1 SQL result = CiBuildTag.connection.exec_query(duplicates_query) puts "Found #{result.count} groups of duplicates." result.each do |group| tag_id = group['tag_id'] build_id = group['build_id'] partition_id = group['partition_id'] ids = group['duplicate_ids'].tr('{}', '').split(',').map(&:to_i) # Keep the first record and delete the rest ids_to_delete = ids[1..-1] # All except the first one puts "For tag_id=#{tag_id}, build_id=#{build_id}, partition_id=#{partition_id}:" puts " Keeping id=#{ids[0]}, deleting ids=#{ids_to_delete.join(', ')}" # Delete the duplicate records CiBuildTag.where(id: ids_to_delete).delete_all end Ci::PendingBuild.all.each_batch do |batch| batch.each do |build| tag_ids = build.tag_ids next unless tag_ids & bad_tags build.tag_ids = tag_ids.map { |x| tag_remap.fetch(x, x) } build.save! end end tagging_models = [Ci::BuildTag, TempRunnerTagging, TempTaggings] tagging_models.each do |tagging_model| tagging_model.include EachBatch; bad_tags.each do |bad_tag_id| tagging_model.where(tag_id: bad_tag_id).each_batch(of: 10000) do |batch| batch.update_all(tag_id: tag_remap.fetch(bad_tag_id)) puts "Updated tag_id #{bad_tag_id} on #{tagging_model.table_name} records to #{tag_remap.fetch(bad_tag_id)}" end end end if Gitlab.ee? ee_tagging_models = [Dast::ProfileTag] tagging_models.concat(ee_tagging_models) ee_tagging_models.each do |tagging_model| tagging_model.all.where(tag_id: bad_tags).each do |row| new_tag_id = tag_remap.fetch(row.tag_id) tagging_model.where(id: row.id).update_all(tag_id: new_tag_id) end end end if tagging_models.any? { |tagging_model| tagging_model.where(tag_id: bad_tag_map.keys).exists? } puts "Something went wrong, bad tags are still in use!" else TempTags.all.where(id: bad_tag_map.keys).each_batch(of: 10000) do |batch| batch.destroy_all end end TempTags.connection.exec_query("DROP INDEX IF EXISTS index_tags_on_name;") TempTags.connection.exec_query("CREATE UNIQUE INDEX index_tags_on_name ON tags USING btree (name);") TempTags.connection.exec_query("REINDEX TABLE p_ci_build_tags;")
Error Message for Script
Loading production environment (Rails 7.0.8.7) irb(main):001:0> load '/tmp/main.rb' D, [2025-05-05T07:02:24.705345 #6064] DEBUG -- : (2.1ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT name, MIN(id) AS id FROM tags GROUP BY name D, [2025-05-05T07:02:24.708593 #6064] DEBUG -- : (1.6ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT id, name FROM tags WHERE id NOT IN (SELECT MIN(id) FROM tags GROUP BY name) {659=>658, 660=>406, 661=>331, 662=>201, 700=>314, 701=>588, 706=>131} D, [2025-05-05T07:02:28.452616 #6064] DEBUG -- : SQL (3741.1ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT tag_id, build_id, partition_id, COUNT(*), ARRAY_AGG(id) as duplicate_ids FROM public.p_ci_build_tags GROUP BY tag_id, build_id, partition_id HAVING COUNT(*) > 1 Found 0 groups of duplicates. D, [2025-05-05T07:02:28.515889 #6064] DEBUG -- : Ci::PendingBuild Load (1.3ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "ci_pending_builds"."id" FROM "ci_pending_builds" ORDER BY "ci_pending_builds"."id" ASC LIMIT 1 D, [2025-05-05T07:02:28.526342 #6064] DEBUG -- : Ci::PendingBuild Load (0.7ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "ci_pending_builds"."id" FROM "ci_pending_builds" WHERE "ci_pending_builds"."id" >= 2008986 ORDER BY "ci_pending_builds"."id" ASC LIMIT 1 OFFSET 1000 D, [2025-05-05T07:02:28.527919 #6064] DEBUG -- : Ci::PendingBuild Load (0.6ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "ci_pending_builds".* FROM "ci_pending_builds" WHERE "ci_pending_builds"."id" >= 2008986 D, [2025-05-05T07:02:28.586340 #6064] DEBUG -- : Feature::FlipperGate Pluck (1.3ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'use_sql_functions_for_primary_key_lookups' D, [2025-05-05T07:02:28.590997 #6064] DEBUG -- : Feature::FlipperGate Pluck (0.5ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'feature_flag_state_logs' D, [2025-05-05T07:02:28.597885 #6064] DEBUG -- : Namespace Load (2.7ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4011 LIMIT 1 D, [2025-05-05T07:02:28.657292 #6064] DEBUG -- : Namespace Load (0.9ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 900 LIMIT 1 D, [2025-05-05T07:02:28.662724 #6064] DEBUG -- : Namespace Load (0.9ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4011 LIMIT 1 D, [2025-05-05T07:02:28.667435 #6064] DEBUG -- : Namespace Load (0.9ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4011 LIMIT 1 D, [2025-05-05T07:02:28.671955 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 3367 LIMIT 1 D, [2025-05-05T07:02:28.677340 #6064] DEBUG -- : Namespace Load (1.6ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 841 LIMIT 1 D, [2025-05-05T07:02:28.682179 #6064] DEBUG -- : Namespace Load (0.9ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.686999 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.691794 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.696190 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.700703 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.705395 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.709688 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.714283 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.718609 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.723031 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.727492 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.731772 #6064] DEBUG -- : Namespace Load (0.7ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.736344 #6064] DEBUG -- : Namespace Load (0.9ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.741055 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.745438 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.749856 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 446 LIMIT 1 D, [2025-05-05T07:02:28.754249 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 2684 LIMIT 1 D, [2025-05-05T07:02:28.758582 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 2684 LIMIT 1 D, [2025-05-05T07:02:28.763180 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4645 LIMIT 1 D, [2025-05-05T07:02:28.767411 #6064] DEBUG -- : Namespace Load (0.7ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 3267 LIMIT 1 D, [2025-05-05T07:02:28.771859 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4645 LIMIT 1 D, [2025-05-05T07:02:28.776331 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4645 LIMIT 1 D, [2025-05-05T07:02:28.780499 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4645 LIMIT 1 D, [2025-05-05T07:02:28.784812 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4645 LIMIT 1 D, [2025-05-05T07:02:28.789400 #6064] DEBUG -- : Namespace Load (0.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:main,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids", "namespaces"."organization_id" FROM "namespaces" WHERE "namespaces"."id" = 4645 LIMIT 1 D, [2025-05-05T07:02:28.796536 #6064] DEBUG -- : Ci::BuildTag Load (1.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "p_ci_build_tags"."id" FROM "p_ci_build_tags" WHERE "p_ci_build_tags"."tag_id" = 659 ORDER BY "p_ci_build_tags"."id" ASC LIMIT 1 D, [2025-05-05T07:02:29.160300 #6064] DEBUG -- : Ci::BuildTag Load (362.5ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "p_ci_build_tags"."id" FROM "p_ci_build_tags" WHERE "p_ci_build_tags"."tag_id" = 660 ORDER BY "p_ci_build_tags"."id" ASC LIMIT 1 D, [2025-05-05T07:02:29.168229 #6064] DEBUG -- : Ci::BuildTag Load (2.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ SELECT "p_ci_build_tags"."id" FROM "p_ci_build_tags" WHERE "p_ci_build_tags"."tag_id" = 660 AND "p_ci_build_tags"."id" >= 1895861 ORDER BY "p_ci_build_tags"."id" ASC LIMIT 1 OFFSET 10000 D, [2025-05-05T07:02:29.176193 #6064] DEBUG -- : Ci::BuildTag Update All (6.8ms) /*application:console,db_config_database:gitlabhq_production,db_config_name:ci,console_hostname:euw1-gitlab-server-22,console_username:rocky*/ UPDATE "p_ci_build_tags" SET "tag_id" = 406 WHERE "p_ci_build_tags"."tag_id" = 660 AND "p_ci_build_tags"."id" >= 1895861 /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params': PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_build_tags_100_tag_id_build_id_partition_id_idx" (ActiveRecord::RecordNotUnique) DETAIL: Key (tag_id, build_id, partition_id)=(406, 3700360, 100) already exists. from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:767:in `block in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:752:in `block in log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:743:in `log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:766:in `exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:745:in `execute_and_clear' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/marginalia-1.11.1/lib/marginalia.rb:91:in `execute_and_clear_with_marginalia' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:67:in `exec_delete' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract/database_statements.rb:175:in `update' ... 40 levels... /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params': ERROR: duplicate key value violates unique constraint "ci_build_tags_100_tag_id_build_id_partition_id_idx" (PG::UniqueViolation) DETAIL: Key (tag_id, build_id, partition_id)=(406, 3700360, 100) already exists. from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:767:in `block in exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:752:in `block in log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activesupport-7.0.8.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract_adapter.rb:743:in `log' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:766:in `exec_no_cache' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql_adapter.rb:745:in `execute_and_clear' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/marginalia-1.11.1/lib/marginalia.rb:91:in `execute_and_clear_with_marginalia' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:67:in `exec_delete' from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract/database_statements.rb:175:in `update' ... 40 levels... irb(main):002:0>
Duplicate TAG Info
SELECT * FROM tags WHERE name IN ('module_test','ipk-build','scl-win','chx','k3','owmc-shell','${RUNNER_TAG}') ORDER BY name;` | id| name| taggings_count| | ------ | ------ |------| |659 | ${RUNNER_TAG} | 0| |658 | ${RUNNER_TAG} | 0| |314 | chx | 0| |700 | chx | 0| |661 | ipk-build | 0| |331 | ipk-build | 0| |588 | k3 | 0| |701 | k3 | 0| |406 | module_test | 0| |660 | module_test | 0| |706 | owmc-shell | 0| |131 | owmc-shell | 0| |201 | scl-win | 0| |662 | scl-win | 0|
Edited by Pedro Pombeiro -
@stanhu Could you please suggest the needful for above error message?
-