Skip to content

Draft: Improve Loose Foreign Keys processing

Omar Qunsulrequested to merge
improve-lfk-tracked-tables into master

What does this MR do and why?

To resolve the issue of LFK worker not handling the high number of deleted p_ci_builds records on CI. I am proposing two changes via this this MR.

  1. If a source table gets empty, we skip this table for the next 30 minutes. To avoid making queries for small or empty batches on the same table again. And focus on the source tables that have so many records instead.
  2. If a connection name has a turbo mode enabled, like ci in this case, we increase its chances to be picked in the next run.

Let me know if these ideas make sense, so that I finalize the MR and write tests.

CC: @ahegyi @morefice

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports