Add workers to clear notified at flags

What does this MR do and why?

implements https://gitlab.com/gitlab-org/gitlab/-/issues/379195+

Checks the past notified groups after 24 hours to see if they are still over_limit?

• If they are still over the limit: reschedule a new check for 24 hours later
• If they not over the limit anymore: clear the notification flag, so they can be notified again if they happen to go over the limit

Note for Infra-Engineers: Implemented with LimitedCapacity::Worker

Screenshots or screen recordings

Manipulating row in rails console, reloading the row and then waiting a bit reading the row again shows it was picked up by the worker and the over limit notified flag was cleared:

Database

The worker uses the following query to fetch new items to work on from the DB. This query will need an index to be created to run efficiently. I suggest we use a similar approach to the OverLimitNotification worker issue where we added an index as a follow-up blocking the feature flag roll out.

Without Index: Fetch a namespace_details item locked for update to work on **Time: 1.603 min**

UPDATE 
  "namespace_details" 
SET 
  "next_over_limit_check_at" = to_timestamp(1678808868) 
WHERE 
  "namespace_details"."namespace_id" IN (
    SELECT 
      "namespace_details"."namespace_id" 
    FROM 
      "namespace_details" 
    WHERE 
      "namespace_details"."free_user_cap_over_limit_notified_at" <= '2023-03-12 15:47:48.266485' 
    ORDER BY 
      "namespace_details"."next_over_limit_check_at" ASC NULLS FIRST 
    LIMIT 
      1 FOR 
    UPDATE 
      SKIP LOCKED
  ) RETURNING *

Time: 1.603 min
  - planning: 1.577 ms
  - execution: 1.603 min
    - I/O read: 1.099 min
    - I/O write: 8.772 s

Shared buffers:
  - hits: 34361 (~268.40 MiB) from the buffer pool
  - reads: 661973 (~5.10 GiB) from the OS file cache, including disk I/O
  - dirtied: 163431 (~1.20 GiB)
  - writes: 42652 (~333.20 MiB)

WITH index: Fetch a namespace_details item locked for update to work on **Time: 3.153 ms**

Query same as above.

Time: 3.153 ms
  - planning: 2.615 ms
  - execution: 0.538 ms
    - I/O read: 0.232 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 3 (~24.00 KiB) from the buffer pool
  - reads: 4 (~32.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

How to set up and validate locally

1. Enable free_user_cap feature flag => Feature.enable :free_user_cap
2. Go to http://localhost:3000/admin/application_settings/general
   1. Set Dashboard enforcement limit to 5
3. Ensure in rails console ::Gitlab::CurrentSettings.dashboard_limit_enabled? is true
4. Enable feature flag free_user_cap_clear_over_limit_notification_flags Feature.enable :free_user_cap_clear_over_limit_notification_flags
5. Create a top-level, free, private group, without a trial
6. Edit the group's namespace details to have a timestamp in free_user_cap_over_limit_notified_at that is more than 24 hours ago and add a timestamp to next_over_limit_check_at that is more than 24 hours ago. (see screenshot)
7. Wait 0-10 min. Latest after the next 5th minute is passed the worker should have picked up your group and removed the timestamp in free_user_cap_over_limit_notified_at since it should not be over the user limit

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.