Activecontext: Log queue_name and preprocessor on processing error

What does this MR do and why?

Adds the queue_name and preprocessor information in the logged fields when an error occurs during the ActiveContext (Semantic Search) indexing process.

This will allow us to correlate errors with their source queue (i.e., the main Code queue or RetryQueue) and preprocessor, so we can understand the full picture of failures.

Code changes

• Queue options
  • ActiveContext::Queue base class:
    • always set the queue_name in preprocess_options
    • add an extra_preprocess_options for child classes to override
  • Have child queue classes override extra_preprocess_options instead of the preprocess_options
• Reference processing
  • In error handling blocks: accept a queue_name and preprocessor, to be logged on errors
  • In fetch_content and apply_embeddings methods:
    • accept a queue_name parameter
    • pass queue_name and preprocessor to error handling blocks
  • In the actual reference class (Ai::ActiveContext::References::Code)
    • make sure the processor blocks accept the queue_name parameter
    • pass the queue_name to the preprocessor methods (fetch_content and apply_embeddings)
  • When items are picked up from a queue for bulk processing, the queue preprocess_options (including queue_name) will be passed to the reference preprocessor (see code)

Process flow

1. The BulkProcessWorker is triggered and calls ActiveContext::BulkProcessQueue.process! for each registered queue
2. ActiveContext::BulkProcessQueue picks up items for processing, making a Reference object out of each item
3. ActiveContext::BulkProcessQueue processes the references with Reference.preprocess_references(refs, **queue.preprocess_options)
   • the queue.preprocess_options will include queue_name
4. The references' preprocessor blocks are ran (defined in Ai::ActiveContext::References::Code)
   • the Ai::ActiveContext::References::Code preprocessors accepts queue_name and passes them to the actual preprocessor methods
   • the preprocessor methods (fetch_content and apply_embeddings) passes the queue_name and preprocessor to the error-handling blocks
   • in the error-handling block, the queue_name and preprocessor are logged on errors

References

• Issue: #604054 (closed)

Screenshots or screen recordings

N/A

How to set up and validate locally

We just need to check that the queue_name and preprocessor fields are logged when an error occurs in the ActiveContext bulk processing pipeline.

1. Ensure that the ActiveContext BulkProcessWorker can run

   • Turn on Semantic Code Search on your GDK instance

   • OR comment out the check preventing the BulkProcessWorker from running:

     diff --git a/gems/gitlab-active-context/lib/active_context/concerns/bulk_async_process.rb b/gems/gitlab-active-context/lib/active_context/concerns/bulk_async_process.rb
     index 723ac5011998..4041c6a1a500 100644
     --- a/gems/gitlab-active-context/lib/active_context/concerns/bulk_async_process.rb
     +++ b/gems/gitlab-active-context/lib/active_context/concerns/bulk_async_process.rb
     @@ -11,7 +11,7 @@ module BulkAsyncProcess
            extend ActiveSupport::Concern
     
            def perform(*args)
     -        return false unless ActiveContext.indexing?
     +        # return false unless ActiveContext.indexing?
     
            if args.empty?
            enqueue_all_shards

2. Add a dummy item to the Code queue:

   ::Ai::ActiveContext::Collections::Code.track_refs!(routing: "1", hashes: ["one"])

3. Run the bulk processing

   ::Ai::ActiveContext::BulkProcessWorker.new.perform("Ai::ActiveContext::Queues::Code", 0)

4. This should result in an error, logged with queue_name and preprocessor information:

   # in GitLab root directory, run:
   tail -f log/active_context.log
   
   # expected error log
   {
     "severity":"WARN",
     "time":"2026-07-16T05:27:08.633Z",
     "correlation_id":"c5bf3282a2deffa90c69c4948bbc470a",
     "exception_class":"<exception class>",
     "exception_message":"<exception message>",
     "exception_backtrace":["backtrace here"],
     "class_name":"Class",
     "queue_name":"code", # new field
     "preprocessor":"content_fetcher", # new field
     "infinite_retry":false, # new field
     "reference":"Ai::ActiveContext::References::Code|5|1|one",
     "reference_id":"one"
   }

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #604054 (closed)