Skip to content

Limit reindexing concurrency

Andreas Brandl requested to merge ab/reindexing-concurrency into master

What does this MR do?

We're going to kick off the reindexing process once per hour from a cronjob. Since reindexing operations can (and will) take longer than that, we use an exclusive lease to acquire a global lock. The idea is to only ever have one reindexing process running at a time.

The approach here is two-fold:

  1. More tightly control the timeout for potentially long lasting statements (DROP/CREATE INDEX mostly) - set this to 6 hours for now
  2. Let the exclusive lease timeout after 24 hours.

The lease timeout should be greater than the maximum time for a reindexing operation to take. Worst case, this performs 3 long lasting statements - so we end up with 3*6 hours for a worst case estimate (give or take).

This isn't perfect of course and we may want to be smarter about this at some point. For now, the intent is to just limit the number of concurrent processes to 1 - for this, this approach should suffice.

Conformity

Edited by Andreas Brandl

Merge request reports