MonitorLockedTablesWorker should not warn about missing tables
We have a worker that checks that all tables that should have a write lock, do actually have a write lock.
This worker however, gives false warnings for tables that do not exist. It is possible that a table exist in the table schema (db/docs
) but the table has not been created yet (because it is created in a post migration)
Impact
This issue can lead to false alerts in #g_tenant-scale Slack channel
Possible root cause
We check that a table is locked by running a query that will return 0 rows for non-existent tables
Solution
Do not include non-existent tables. Not sure what the best place is to exclude non-existent tables.
Steps to reproduce:
Using local development (gdk):
- Ensure all write locks are set
bundle exec rake gitlab:db:lock_writes
- Create a file in
db/docs/new_table.yml
with this content:--- table_name: new_table gitlab_schema: gitlab_main_cell
- In a rails console, run
Database::MonitorLockedTablesWorker.new.perform
Expected output
No tables should need locking
{"main"=>{:tables_need_lock=>[], :tables_need_lock_count=>0, :tables_need_unlock=>[], :tables_need_unlock_count=>0},
"ci"=>{:tables_need_lock=>[], :tables_need_lock_count=>1, :tables_need_unlock=>[], :tables_need_unlock_count=>0}}
Actual output
The non-existent table new_table
is included in the output:
{"main"=>{:tables_need_lock=>[], :tables_need_lock_count=>0, :tables_need_unlock=>[], :tables_need_unlock_count=>0},
"ci"=>{:tables_need_lock=>["new_table"], :tables_need_lock_count=>1, :tables_need_unlock=>[], :tables_need_unlock_count=>0}}
Edited by Rutger Wessels