Skip to content

fix(chat): authorization and telemetry of duo chat tools

Shinya Maeda requested to merge authorize-injected_context into main

What does this merge request do and why?

This MR fixes the following bugs of Duo Chat:

In addition, tests are added to prevent regressions from happening in the future development. No changes on the business logic, including prompts.

Closes Authorization and Telemetry for include_<catego... (gitlab-org/gitlab#491598 - closed)

Related to Add unit primitive for injected context (gitlab-org/gitlab!168327 - merged)

How to set up and validate locally

Request 1:

curl -X 'POST' \
  'http://localhost:5052/v2/chat/agent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -H 'x-gitlab-enabled-feature-flags: expanded_ai_logging' \
  -d '{
  "messages": [
    {"role": "user", "content": "Summarize the issue #123"}
  ],
  "options": {
    "agent_scratchpad": {
      "agent_type": "react",
      "steps": []
    } 
  }
}'

Access log 1:

{"url": "http://localhost:5052/v2/chat/agent", "path": "/v2/chat/agent", "status_code": 200, "method": "POST", "correlation_id": "2044d45d5716452ea977792d8dcb9a20", "http_version": "1.1", "client_ip": "127.0.0.1", "client_port": 54686, "duration_s": 2.7029943720008305, "duration_request": -1, "request_arrived_at": "2024-10-15T04:50:00.949466+00:00", "response_start_duration_s": 0.19052534399997967, "first_chunk_duration_s": 2.7023798690006515, "cpu_s": 0.23548293899999972, "content_type": "application/x-ndjson; charset=utf-8", "user_agent": "curl/7.81.0", "gitlab_language_server_version": null, "gitlab_instance_id": null, "gitlab_global_user_id": null, "gitlab_host_name": null, "gitlab_version": null, "gitlab_saas_duo_pro_namespace_ids": null, "gitlab_realm": null, "gitlab_duo_seat_count": null, "enabled_feature_flags": "expanded_ai_logging", "meta.feature_category": "duo_chat", "duo_chat.agent_available_tools": ["ci_editor_assistant", "gitlab_documentation", "epic_reader", "issue_reader", "merge_request_reader"], "duo_chat.agent_tool_action": "issue_reader", "tracked_internal_events": ["request_duo_chat", "request_ask_issue"], "logger": "api.access", "level": "info", "type": "mlops", "stage": "main", "timestamp": "2024-10-15T04:50:03.652564Z", "message": "127.0.0.1:54686 - \"POST /v2/chat/agent HTTP/1.1\" 200"}

Request 2:

curl -X 'POST' \
  'http://localhost:5052/v2/chat/agent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -H 'x-gitlab-enabled-feature-flags: expanded_ai_logging' \
  -d '{
  "messages": [
    {"role": "user", "content": "How can I create an issue in GitLab?"}
  ],
  "options": {
    "agent_scratchpad": {
      "agent_type": "react",
      "steps": []
    } 
  }
}'

Access log 2:

{"url": "http://localhost:5052/v2/chat/agent", "path": "/v2/chat/agent", "status_code": 200, "method": "POST", "correlation_id": "fd499f716b1b4d09ad6d062e66ddaf4b", "http_version": "1.1", "client_ip": "127.0.0.1", "client_port": 38626, "duration_s": 3.1050202089991217, "duration_request": -1, "request_arrived_at": "2024-10-15T04:50:32.203492+00:00", "response_start_duration_s": 0.004367227000329876, "first_chunk_duration_s": 3.1045370329993602, "cpu_s": 0.041988223999999796, "content_type": "application/x-ndjson; charset=utf-8", "user_agent": "curl/7.81.0", "gitlab_language_server_version": null, "gitlab_instance_id": null, "gitlab_global_user_id": null, "gitlab_host_name": null, "gitlab_version": null, "gitlab_saas_duo_pro_namespace_ids": null, "gitlab_realm": null, "gitlab_duo_seat_count": null, "enabled_feature_flags": "expanded_ai_logging", "meta.feature_category": "duo_chat", "duo_chat.agent_available_tools": ["ci_editor_assistant", "gitlab_documentation", "epic_reader", "issue_reader", "merge_request_reader"], "duo_chat.agent_tool_action": "gitlab_documentation", "tracked_internal_events": ["request_documentation_search", "request_duo_chat"], "logger": "api.access", "level": "info", "type": "mlops", "stage": "main", "timestamp": "2024-10-15T04:50:35.308566Z", "message": "127.0.0.1:38626 - \"POST /v2/chat/agent HTTP/1.1\" 200"}

Request 3:

curl -X 'POST' \
  'http://localhost:5052/v2/chat/agent' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -H 'x-gitlab-enabled-feature-flags: expanded_ai_logging' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "How can I log output?",
      "additional_context": [
        { "id": "123", "category": "merge_request", "content": "something", "metadata": {"a": "b"}}
      ]
    },
    {"role": "assistant", "content": "Use print function"},
    {
      "role": "user",
      "content": "How can I log output?",
      "context": { "type": "issue", "content": "This is awesome issue" },
      "current_file": { "file_path": "main.py", "data": "print", "selected_code": true },
      "additional_context": [
        { "id": "456", "category": "file", "content": "helloworld", "metadata": {"c": "d"}}
      ]
    }
  ],
  "options": {
    "agent_scratchpad": {
      "agent_type": "react",
      "steps": []
    }
  }
}'

Access log 3:

{"url": "http://localhost:5052/v2/chat/agent", "path": "/v2/chat/agent", "status_code": 200, "method": "POST", "correlation_id": "895fb42c6ba94de3a7ae4d92c7b029dc", "http_version": "1.1", "client_ip": "127.0.0.1", "client_port": 44692, "duration_s": 8.770223927998813, "duration_request": -1, "request_arrived_at": "2024-10-15T04:51:00.641311+00:00", "response_start_duration_s": 0.006810416998632718, "first_chunk_duration_s": 3.233347760999095, "cpu_s": 0.15046201500000045, "content_type": "application/x-ndjson; charset=utf-8", "user_agent": "curl/7.81.0", "gitlab_language_server_version": null, "gitlab_instance_id": null, "gitlab_global_user_id": null, "gitlab_host_name": null, "gitlab_version": null, "gitlab_saas_duo_pro_namespace_ids": null, "gitlab_realm": null, "gitlab_duo_seat_count": null, "enabled_feature_flags": "expanded_ai_logging", "meta.feature_category": "duo_chat", "duo_chat.agent_available_tools": ["ci_editor_assistant", "gitlab_documentation", "epic_reader", "issue_reader", "merge_request_reader"], "tracked_internal_events": ["request_duo_chat", "request_include_file_context", "request_include_merge_request_context"], "logger": "api.access", "level": "info", "type": "mlops", "stage": "main", "timestamp": "2024-10-15T04:51:09.411588Z", "message": "127.0.0.1:44692 - \"POST /v2/chat/agent HTTP/1.1\" 200"}

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Shinya Maeda

Merge request reports