Refactor Reenqueuer to not sleep

The following discussion from !22031 (merged) should be addressed:

• @mikolaj_wawrzyniak started a discussion: (+4 comments)

  I'm still not convinced for usage of sleep here. It feel wasteful, to put process to sleep. I wonder maybe we can try to use lease mechanism here as well.

Possible solution

• Reenqueue at a specific time immediately after the end of minimum duration.
• Take another exclusive lease, open-ended, which expires exactly at the end of minimum duration.

Note:

Sidekiq scheduled jobs introduces another delay that averages to the poll interval.

Checking for New Jobs

Sidekiq's scheduler is not meant to be second-precise. It checks for scheduled jobs approximately every 5 seconds by default (15 seconds before Sidekiq 5.1)

A case where this matters:

• There is a ton of work to do (backfill condition), therefore we don't really want to insert a lot of delay between jobs.
• The job is performant, and even with maximum work to do, it always completes below minimum duration. Great!

If we ignore poll interval mechanics, we might think that with a minimum interval of 5s, the job will run every ~5s. But with a default poll interval of 5s, it would actually run closer to every 10s.

Therefore we should subtract average_scheduled_poll_interval from minimum_interval. If the wait duration is < 0, then reenqueue without a target time.