Add ability to stream chat responses

What does this MR do and why?

This adds the capabilities to stream messages for GitLab Duo chat.

The streamed chunks get send over aiCompletionResponse but the order is not guaranteed on the client. Each chunk has a chunkId which has a guaranteed order, starting with 1.

The client_subscription_id is meant for the client to identify which mutation belongs to which subscription response. However, chat is a special feature where we want to sync responses between multiple chat instances of the same user.

This means we want to have two different subscriptions for the same request: 1. User-specific to sync between multiple clients 2. Request-specific for streaming

By not streaming the response when a client_subscription_id is not present, we also make sure that older clients still work when the stream_gitlab_duo is enabled. Otherwise, they would receive a streamed response on a subscription on user_id and resource_id.

NOTE: Related FE MR by @dmishunov !130347 (merged)

Screenshots or screen recordings

The following only works with the FE and graphql changes. There is an MR that combines both: !130415 (closed)

Without FF	With FF
streaming-example-no-ff	streaming-example

Before	After

How to set up and validate locally

1. Have GitLab Duo and all AI features set up
2. Enable the FF stream_gitlab_duo
3. Checkout !130415 (closed) as it combines this MR with the FE changes and updates the GraphQL subscriptions

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

ee/app/assets/javascripts/ai/tanuki_bot/store/mutations.js

@@ -10,6 +10,7 @@ export default {
       }
 
       if (newMessageData.chunkId) {
+        // when a new chunk comes in, we append the message if it already exists
         const index = state.messages.findIndex(
           (msg) =>
             msg.requestId === newMessageData.requestId &&
@@ -17,8 +18,14 @@ export default {
         );
         if (index > -1) {
           const prevMessage = state.messages[index];
+
+          // When the final response comes before the last chunk
+          if (prevMessage.chunkId === null) {
+            return;
+          }
+
           const newMessage = {
-            ...prevMessage,
+            ...newMessageData,
             content: prevMessage.content.concat(newMessageData.content),
           };
           state.messages.splice(index, 1, newMessage);
@@ -27,11 +34,7 @@ export default {
           state.loading = false;
         }
       } else {
-        const index = state.messages.findIndex(
-          (msg) =>
-            msg.requestId === newMessageData.requestId &&
-            msg.role.toLowerCase() === newMessageData.role.toLowerCase(),
-        );
+        const index = state.messages.findIndex((msg) => msg.requestId === newMessageData.requestId);
         const hasMsgWithRequestId = index > -1;
         const msgWithRequestId = hasMsgWithRequestId && state.messages[index];
         let isLastMessage = false;
@@ -39,14 +42,27 @@ export default {
         if (hasMsgWithRequestId) {
           if (msgWithRequestId.role.toLowerCase() === newMessageData.role.toLowerCase()) {
             // We update the existing message object instead of pushing a new one
-            state.messages[index] = {
-              ...msgWithRequestId,
+
+            const newMessage = {
+              ...state.messages[index],
               ...newMessageData,
             };
+            state.messages.splice(index, 1, newMessage);
           } else {
-            // We add the new ASSISTANT message
-            isLastMessage = index === state.messages.length - 1;
-            state.messages.splice(index + 1, 0, newMessageData);
+            // We add the new ASSISTANT message. We check first if one already exists.
+            // This can happen when a the ASSISTANT message already got streamed
+            const assistantIndex = state.messages.findIndex(
+              (msg) =>
+                msg.requestId === newMessageData.requestId &&
+                msg.role.toLowerCase() === GENIE_CHAT_MODEL_ROLES.assistant.toLocaleLowerCase(),
+            );
+
+            if (assistantIndex > -1) {
+              state.messages.splice(assistantIndex, 1, newMessageData);
+            } else {
+              isLastMessage = index === state.messages.length - 1;
+              state.messages.splice(index + 1, 0, newMessageData);
+            }
           }
         } else {
           // It's the new message, so just push it to the end of the Array