Refactor index_over_watermark_check in zoekt scheduling service

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

The following discussion from !172790 (merged) should be addressed:

  • @maddievn started a discussion: (+1 comment)

    meta question: Should we rethink this event a bit now that we no longer use the watermark level attribute in the event? We can add a scope called :should_update_watermark_level which returns all indices that need to be updated (instead of one event per watermark level) in index_over_watermark_check and this handler simply calls update_reserved_storage_bytes for the indices. Let me know what you think!

Edited by 🤖 GitLab Bot 🤖