Allow gem to register queues

What does this MR do and why?

Similar to the queues we have for elasticsearch currently, the ActiveContext gem will use queues for tracking and processing refs. This MR is to register the queues.

This MR introduces a concern that can be added to the monolith for registering a queue. All registered queues will bulk process refs like the bulk cron worker does.

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

• Register queues (#507971 - closed)
• Figjam link

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

To create a new queue: add a file in the monolith, extend ActiveContext::Concerns::Queue, define number_of_shards and call register!:

# frozen_string_literal: true

module Ai
  module Context
    module Queues
      class MergeRequest
        extend ActiveContext::Concerns::Queue

        def self.number_of_shards
          2
        end

        register!
      end
    end
  end
end

To access the unique queues:

ActiveContext.queues
=> #<Set: {"ai_context_queues:{merge_request}"}>

To view sharded queues:

ActiveContext.raw_queues
=> ["ai_context_queues:{merge_request}:0", "ai_context_queues:{merge_request}:1"]

Related to #507971 (closed)