Skip to content

Let TableLocker return hash with results

Rutger Wessels requested to merge rw-table-locker-dry-run-hash into master

What does this MR do and why?

This MR is related to gitlab-com/gl-infra/production#8564 (closed)

We want to run rake gitlab:database:lock_writes on production another time. This rake task has a 'dry-run' option, that we can use to inspect what will happen. We can then see detailed logging.

However, we need more concise information:

  • Only list the tables that are (about to be) locked
  • List number of tables that will:
    • be locked (for ci and main)
    • unlocked (for ci and main)
    • skipped (for ci and main)

This PR will make Gitlab::Database::TablesLocker.new(dry_run: true).lock_writes return an array with hashes. Each hash has:

  • Key 'table'
  • Key 'action' (skipped, locked, unlocked)
  • Key 'database' (ci/main)
  • Key 'dry_run' (true/false)

Example return value:

results = Gitlab::Database::TablesLocker.new(dry_run: true).lock_writes
[
 {:action=>"locked", :database=>"ci", :table=>"achievements", :dry_run=>true},
 {:action=>"skipped", :database=>"ci", :table=>"agent_activity_events", :dry_run=>true},
 {:action=>"skipped", :database=>"ci", :table=>"agent_group_authorizations", :dry_run=>true},
 {:action=>"skipped", :database=>"ci", :table=>"agent_project_authorizations", :dry_run=>true},
 {:action=>"skipped", :database=>"ci", :table=>"alert_management_alert_assignees", :dry_run=>true}
]

Before we run the rake task for locking the tables, we can now more easily assess what will happen. On production or staging, using a Rails Console, we can run this snippets to get a summary and understand the impact of running the rake gitlab:database:lock_writes before we actually run it:

results = Gitlab::Database::TablesLocker.new(dry_run: true).lock_writes

summary = {
  tables_locked: {},
  nr_locked: {},
  nr_skipped: {},
  nr_unlocked: {}
}

results.each do |result|
  total_key = "nr_#{result[:action]}".to_sym
  summary[total_key][result[:database]] ||= 0
  summary[:tables_locked][result[:database]] ||= []

  summary[:tables_locked][result[:database]] << result[:table] if result[:action] == 'locked'
  summary[total_key][result[:database]] += 1
end

pp summary

Which could result in an output like this:

  • Tables locked: for each database, a list of tables that have been locked
  • nr_locked: Number of tables that have been locked
  • nr_skipped: Number of tables that are already locked and therefore skipped
  • nr_unlocked: Number of tables that are unlocked (this happens regardless of current lock status)
{:tables_locked=>
  {"main"=>
    ["ci_pipeline_metadata",
     "ci_runner_machines",
     "p_ci_builds_metadata",
     "gitlab_partitions_dynamic.ci_builds_metadata_101",
     "gitlab_partitions_dynamic.ci_runner_machine_builds_100"],
   "ci"=>
    ["gitlab_partitions_static.analytics_cycle_analytics_issue_stage_events_00",
     <SNIP>...
     "work_item_hierarchy_restrictions",
     "work_item_progresses"]},
 :nr_locked=>{"main"=>5, "ci"=>383},
 :nr_skipped=>{"main"=>58, "ci"=>534},
 :nr_unlocked=>{"main"=>950, "ci"=>96}}

Full output: here

How to set up and validate locally

  1. Start Rails console (using gdk: gdk rails c)
  2. In Rails console, run
    results = Gitlab::Database::TablesLocker.new(dry_run: true).lock_writes
  3. results will be an array. Each element is an hash that refers to a table and what happens to that table.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Rutger Wessels

Merge request reports