Do not render chunks once full message is received
What does this MR do and why?
We do not guarantee the order when subscription messages are received. This is the case for chunks, but also for the full message.
So it could happen that we receive the full message from aiCompletion
but then still receive one or more chunks. In this case we should stop
trying to compute the full message based on the received chunks and
ignore the chunk. Otherwise this would lead to a bug where the user only
sees the last chunk rendered.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
before | after |
How to set up and validate locally
This is hard to replicate as it fully relies on the order of the streamed messages that is not guaranteed. With the following script, you can replicate it:
- Use a fixed
clientSubscriptionId
to send the message to
diff --git a/ee/app/assets/javascripts/ai/tanuki_bot/components/app.vue b/ee/app/assets/javascripts/ai/tanuki_bot/components/app.vue
index 59ad49a90c11..ceed972d4e24 100644
--- a/ee/app/assets/javascripts/ai/tanuki_bot/components/app.vue
+++ b/ee/app/assets/javascripts/ai/tanuki_bot/components/app.vue
@@ -76,7 +76,7 @@ export default {
return {
userId: this.userId,
resourceId: this.resourceId || this.userId,
- clientSubscriptionId: this.clientSubscriptionId,
+ clientSubscriptionId: '123',
htmlResponse: false,
};
},
diff --git a
- Run the following script to get the last chunk out of order
chunk1 = Gitlab::Llm::AiMessage.for(action: :chat).new({user: User.first, resource: User.first, content: "hello", role: Gitlab::Llm::AiMessage::ROLE_ASSISTANT, client_subscription_id: '123', request_id: 'foobar', chunk_id: 1})
chunk2 = Gitlab::Llm::AiMessage.for(action: :chat).new({user: User.first, resource: User.first, content: " world", role: Gitlab::Llm::AiMessage::ROLE_ASSISTANT, client_subscription_id: '123', request_id: 'foobar', chunk_id: 2})
full_message = Gitlab::Llm::AiMessage.for(action: :chat).new({user: User.first, resource: User.first, content: "hello world.", role: Gitlab::Llm::AiMessage::ROLE_ASSISTANT, client_subscription_id: '123', request_id: 'foobar', chunk_id: nil})
chunk3 = Gitlab::Llm::AiMessage.for(action: :chat).new({user: User.first, resource: User.first, content: ".", role: Gitlab::Llm::AiMessage::ROLE_ASSISTANT, client_subscription_id: '123', request_id: 'foobar', chunk_id: 3})
GraphqlTriggers.ai_completion_response(chunk1)
GraphqlTriggers.ai_completion_response(chunk2)
GraphqlTriggers.ai_completion_response(full_message)
GraphqlTriggers.ai_completion_response(chunk3)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.