Update container expiration policy worker [RUN ALL RSPEC] [RUN AS-IF-FOSS]

🎱 Context

Cleanup policies are tasks executed by a background worker. We're not going to detail the task here but it's a non trivial process.

Currently, each enabled cleanup policy ready to run is read by a cron worker. This cron worker will:

1. "Mark" container repositories
2. Enqueue jobs for the related limited capacity worker

The main problem is that (1.) is not scalable. The SQL query execute to get all the runnable policies is quite complex. In addition, the worker loops on each row to then mark the linked container repositories.

This is issue #267546 (closed).

The found solution is detailed in #267546 (closed).

Since we're changing the whole logic for both workers, this effort is splitted in steps and gated behind a feature flag so that we can "rollback" to the current logic at will:

1. (You are here ⛳) Introduce the feature flag + update the cron worker according to #267546 (closed)
2. Prepare the limited worker for changes.
3. Update the limited worker to support the new logic

🔍 What does this MR do?

• Introduces the container_registry_expiration_policies_loopless feature flag.
  • Rollout issue: #325273 (closed)
• Update ContainerExpirationPolicyWorker (the cron worker) so that:
  • if the feature flag is disabled, follow the "old" behavior (including looping through policies)
  • if the feature flag is enabled
    • do not loop through policies. Don't even query them
  • keep the rest as it is. In particular, use the exclusive lease for both cases.
• Update the related specs

🚫 What this MR does not do

• Update the limited worker (that's step (2.) and (3.))

Screenshots (strongly suggested)

n / a

📏 Does this MR meet the acceptance criteria?

Conformity

• 📋 Does this MR need a changelog?
  • I have included a changelog entry.
  • I have not included a changelog entry because the changes are behind a feature flag.
• [-] Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• [-] Separation of EE specific content

Availability and Testing

• [-] Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• [-] Tested in all supported browsers
• [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• [-] Label as security and @ mention @gitlab-com/gl-security/appsec
• [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• [-] Security reports checked/validated by a reviewer from the AppSec team