Introduce a new logger/log file for ActiveContext

What does this MR do and why?

We are introducing a new indexing pipeline for Code Embeddings (see blueprint). Currently, this pipeline is using a basic ::Logger.

In this MR, we introduce a new JsonLogger with its own log file for ActiveContext following the guide in https://docs.gitlab.com/development/logging/#how-to-use-json-logging. This allows us to isolate anything related to ActiveContext to its own logs.

A note on MR scope

We are not adding new logging here yet. We are simply redirecting existing logging to activecontext.log.

Essentially, this covers any logging under:

  • ee/lib/ai/active_context/
  • gems/gitlab-active-context/

For Next Steps, we will add logging to:

  • ee/app/services/ai/active_context/code/initial_indexing_service.rb
  • ee/app/services/ai/active_context/code/indexer.rb

References

Screenshots or screen recordings

N/A

How to set up and validate locally

  1. Follow the setup in #550418 (comment 2610944159)

  2. After running/processing the embeddings queue (ActiveContext.execute_all_queues!), you should see a new log in <gitlab_dir>/log for activecontext.log

  3. You should see these kinds of INFO logs:

    # in gitlab root directory
    > tail -f log/activecontext.log
    {"severity":"INFO","time":"2025-07-16T07:34:14.565Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexing_end","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":21.0,"meta.indexing.last_score":30.0,"meta.indexing.failures_count":0,"meta.indexing.bulk_execution_duration_s":2.7109440001659095}
    {"severity":"INFO","time":"2025-07-16T07:38:22.275Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","queue":"Ai::ActiveContext::Queues::Code","message":"bulk_indexing_start","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":31.0,"meta.indexing.last_score":40.0}
    {"severity":"INFO","time":"2025-07-16T07:38:24.568Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","message":"bulk_submitted","meta.indexing.bulk_count":10,"meta.indexing.errors_count":0}
    {"severity":"INFO","time":"2025-07-16T07:38:24.568Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexer_flushed","meta.indexing.flushing_duration_s":0.04447200009599328}
    {"severity":"INFO","time":"2025-07-16T07:38:24.568Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexing_end","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":31.0,"meta.indexing.last_score":40.0,"meta.indexing.failures_count":0,"meta.indexing.bulk_execution_duration_s":2.2963870000094175}
    {"severity":"INFO","time":"2025-07-16T07:59:41.291Z","queue":"Ai::ActiveContext::Queues::Code","message":"bulk_indexing_start","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":81.0,"meta.indexing.last_score":90.0}
    {"severity":"INFO","time":"2025-07-16T07:59:45.103Z","correlation_id":"07742e712b17e0a4ef541c5c7577e503","message":"bulk_submitted","meta.indexing.bulk_count":10,"meta.indexing.errors_count":0}
    {"severity":"INFO","time":"2025-07-16T07:59:45.103Z","correlation_id":"07742e712b17e0a4ef541c5c7577e503","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexer_flushed","meta.indexing.flushing_duration_s":0.023725999984890223}
    {"severity":"INFO","time":"2025-07-16T07:59:45.103Z","correlation_id":"07742e712b17e0a4ef541c5c7577e503","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexing_end","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":81.0,"meta.indexing.last_score":90.0,"meta.indexing.failures_count":0,"meta.indexing.bulk_execution_duration_s":3.8154360000044107}

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #554925 (closed)

Edited by Pam Artiaga

Merge request reports

Loading