Introduce a new logger/log file for ActiveContext
What does this MR do and why?
We are introducing a new indexing pipeline for Code Embeddings (see blueprint). Currently, this pipeline is using a basic ::Logger.
In this MR, we introduce a new JsonLogger with its own log file for ActiveContext following the guide in https://docs.gitlab.com/development/logging/#how-to-use-json-logging. This allows us to isolate anything related to ActiveContext to its own logs.
A note on MR scope
We are not adding new logging here yet. We are simply redirecting existing logging to activecontext.log.
Essentially, this covers any logging under:
ee/lib/ai/active_context/gems/gitlab-active-context/
For Next Steps, we will add logging to:
ee/app/services/ai/active_context/code/initial_indexing_service.rbee/app/services/ai/active_context/code/indexer.rb
References
- Issue: #554925 (closed)
- Logging discussion: #545063 (comment 2618954132)
Screenshots or screen recordings
N/A
How to set up and validate locally
-
Follow the setup in #550418 (comment 2610944159)
-
After running/processing the embeddings queue (
ActiveContext.execute_all_queues!), you should see a new log in<gitlab_dir>/logforactivecontext.log -
You should see these kinds of INFO logs:
# in gitlab root directory > tail -f log/activecontext.log {"severity":"INFO","time":"2025-07-16T07:34:14.565Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexing_end","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":21.0,"meta.indexing.last_score":30.0,"meta.indexing.failures_count":0,"meta.indexing.bulk_execution_duration_s":2.7109440001659095} {"severity":"INFO","time":"2025-07-16T07:38:22.275Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","queue":"Ai::ActiveContext::Queues::Code","message":"bulk_indexing_start","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":31.0,"meta.indexing.last_score":40.0} {"severity":"INFO","time":"2025-07-16T07:38:24.568Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","message":"bulk_submitted","meta.indexing.bulk_count":10,"meta.indexing.errors_count":0} {"severity":"INFO","time":"2025-07-16T07:38:24.568Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexer_flushed","meta.indexing.flushing_duration_s":0.04447200009599328} {"severity":"INFO","time":"2025-07-16T07:38:24.568Z","correlation_id":"611591135978e63595fa70aa5eb5e08c","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexing_end","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":31.0,"meta.indexing.last_score":40.0,"meta.indexing.failures_count":0,"meta.indexing.bulk_execution_duration_s":2.2963870000094175} {"severity":"INFO","time":"2025-07-16T07:59:41.291Z","queue":"Ai::ActiveContext::Queues::Code","message":"bulk_indexing_start","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":81.0,"meta.indexing.last_score":90.0} {"severity":"INFO","time":"2025-07-16T07:59:45.103Z","correlation_id":"07742e712b17e0a4ef541c5c7577e503","message":"bulk_submitted","meta.indexing.bulk_count":10,"meta.indexing.errors_count":0} {"severity":"INFO","time":"2025-07-16T07:59:45.103Z","correlation_id":"07742e712b17e0a4ef541c5c7577e503","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexer_flushed","meta.indexing.flushing_duration_s":0.023725999984890223} {"severity":"INFO","time":"2025-07-16T07:59:45.103Z","correlation_id":"07742e712b17e0a4ef541c5c7577e503","class":"ActiveContext::BulkProcessQueue","message":"bulk_indexing_end","meta.indexing.redis_set":"ai_activecontext_queues:{code}:0:zset","meta.indexing.refs_count":10,"meta.indexing.first_score":81.0,"meta.indexing.last_score":90.0,"meta.indexing.failures_count":0,"meta.indexing.bulk_execution_duration_s":3.8154360000044107}
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #554925 (closed)