Add llm log events to docs
What does this MR do and why?
It documents every event logged to llm.log
file and updates documentation on logging to reflect recent changes in this MR
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
Merge request reports
Activity
assigned to @mksionek
- A deleted user
1 Warning This merge request is quite big (734 lines changed), please consider splitting it into multiple merge requests. 1 Message This MR contains docs in the /doc/development directory, but any Maintainer (other than the author) can merge. You do not need tech writer review. If needed, you can retry the
danger-review
job that generated this comment.Generated by
DangerEdited by Ghost Useradded docs-only label
changed milestone to %17.5
requested review from @sselhorn
mentioned in epic gitlab-org#15150 (closed)
- Resolved by Suzanne Selhorn
- Resolved by Suzanne Selhorn
- Resolved by Suzanne Selhorn
- Resolved by Suzanne Selhorn
Hi @mksionek
You are testing the limits of what our docs site is capable of! Take a look at the rendered content: https://main-ee-167304.docs.gitlab-review.app/ee/development/ai_features/logged_events.htmlI'm on my laptop, so my screen is a bit narrower, but I see things overwriting each other and going off the right side of the screen.
Could we separate the information out by one or more criteria and have multiple tables instead? I just gave the table to Claude and asked for advice, and here's what it said. Does this make any sense? I'm open to ideas, I think we just need a few fewer columns.
Event Logging Tables
Table 1: Event Basic Information
Event message description class ai_event_name level Returning from Service due to validation user not permitted to perform action Llm::BaseService
permission_denied info Enqueuing CompletionWorker scheduling completion worker in sidekiq Llm::BaseService
worker_enqueued info aborting: missing resource If there is no resource for slash command Llm::ChatService
missing_resource info Performing CompletionService performing completion Llm::Internal::CompletionService
completion_service_performed info Answer from LLM response Get answer from response Gitlab::Llm::Chain::Answer
answer_received info Final answer Get final answer from response Gitlab::Llm::Chain::Answer
final_answer_received info Default final answer Default final answer: I'm sorry, I couldn't respond in time. Please try a more specific request or enter /clear to start a new chat. Gitlab::Llm::Chain::Answer
default_final_answer_received info Error message/ "Error" when answering with an error Gitlab::Llm::Chain::Answer
error_returned error Table 2: Logging Details
Event message Logged with monolith logs Logged with sidekiq logs Part of the system Expanded logging? Returning from Service due to validation yes no abstraction_layer no Enqueuing CompletionWorker yes no abstraction_layer yes aborting: missing resource yes no duo_chat no Performing CompletionService no yes abstraction_layer yes Answer from LLM response no yes duo_chat yes Final answer no yes duo_chat yes Default final answer no yes duo_chat yes Error message/ "Error" no yes duo_chat no Table 3: Additional Event Information
Event message arguments error_code Returning from Service due to validation none - Enqueuing CompletionWorker user_id: message.user.id, resource_id: message.resource&.id, resource_class: message.resource&.class&.name, request_id: message.request_id, action_name: message.ai_action, options: job_options - aborting: missing resource none - Performing CompletionService user_id: prompt_message.user.to_gid, resource_id: prompt_message.resource&.to_gid, action_name: prompt_message.ai_action, request_id: prompt_message.request_id, client_subscription_id: prompt_message.client_subscription_id, completion_service_name: completion_class_name - Answer from LLM response llm_answer_content: content - Final answer llm_answer_content: content - Default final answer - "A6000" Error message/ "Error" error: content, error_code: error_code, source: source - Edited by Suzanne Selhorn