Skip to content

Show more specific errors for Explain This Vulnerability feature

Daniel Tian requested to merge 408343-show-more-specific-errors into master

What does this MR do and why?

This MR shows more specific error messages for the Explain This Vulnerability feature, instead of the generic "An error has occurred" message. It shows errors from two different places:

  • Error with the request, from the aiAction mutation:
    1. When the user goes over the rate limit.
    2. When the resourceId sent to the aiAction not in the correct format.
    3. When the resourceId sent to the aiAction does not exist.
    4. Any other backend error that occurs for the mutation.
  • Error with the response, from the aiCompletionResponse subscription:
    1. When there is no OpenAI API key set.
    2. Any errors from the OpenAI API.
    3. Any errors while processing the response from OpenAI.
ksnip_20230424-161553

How to set up and validate locally

Apply the patches to reproduce the errors shown in the screenshots.

Rate limit error:

Index: lib/gitlab/application_rate_limiter.rb
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/lib/gitlab/application_rate_limiter.rb b/lib/gitlab/application_rate_limiter.rb
--- a/lib/gitlab/application_rate_limiter.rb	(revision 4670eea920cf758ac6532068314264f59be93a49)
+++ b/lib/gitlab/application_rate_limiter.rb	(date 1682379669510)
@@ -57,7 +57,7 @@
           namespace_exists: { threshold: 20, interval: 1.minute },
           fetch_google_ip_list: { threshold: 10, interval: 1.minute },
           project_fork_sync: { threshold: 10, interval: 30.minutes },
-          ai_action: { threshold: 160, interval: 8.hours },
+          ai_action: { threshold: 1, interval: 8.hours },
           jobs_index: { threshold: 600, interval: 1.minute },
           bulk_import: { threshold: 6, interval: 1.minute },
           projects_api_rate_limit_unauthenticated: {
ksnip_20230424-162346

resourceId sent to the aiAction not in the correct format:

Index: ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue b/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue
--- a/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue	(revision 4670eea920cf758ac6532068314264f59be93a49)
+++ b/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue	(date 1682389705613)
@@ -88,7 +88,7 @@
       return getContentWrapperHeight(DRAWER_CONTAINER_CLASS);
     },
     vulnerabilityGraphqlId() {
-      return convertToGraphQLId(TYPENAME_VULNERABILITY, this.vulnerability.id);
+      return convertToGraphQLId(TYPENAME_VULNERABILITY, '123');
     },
   },
   methods: {
ksnip_20230424-162715

resourceId sent to the aiAction does not exist:

Index: ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue b/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue
--- a/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue	(revision 4670eea920cf758ac6532068314264f59be93a49)
+++ b/ee/app/assets/javascripts/vulnerabilities/components/explain_this_vulnerability.vue	(date 1682389688437)
@@ -88,7 +88,7 @@
       return getContentWrapperHeight(DRAWER_CONTAINER_CLASS);
     },
     vulnerabilityGraphqlId() {
-      return convertToGraphQLId(TYPENAME_VULNERABILITY, this.vulnerability.id);
+      return convertToGraphQLId(TYPENAME_VULNERABILITY, this.vulnerability.id + 9999999);
     },
   },
   methods: {
ksnip_20230424-162916

When there is no OpenAI API key set:

Run the following command in a terminal:

echo 'Gitlab::CurrentSettings.update(openai_api_key: "")' | rails c
ksnip_20230424-163813

When there is an error from OpenAPI:

Run the following command in a terminal:

echo 'Gitlab::CurrentSettings.update(openai_api_key: "abcdefghijklmnopqrstuvwxyz")' | rails c
ksnip_20230424-164037

Error when processing OpenAI response:

Index: ee/lib/gitlab/llm/open_ai/completions/explain_vulnerability.rb
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/ee/lib/gitlab/llm/open_ai/completions/explain_vulnerability.rb b/ee/lib/gitlab/llm/open_ai/completions/explain_vulnerability.rb
--- a/ee/lib/gitlab/llm/open_ai/completions/explain_vulnerability.rb	(revision 4773b8ff2b96989d6a31ae56c4848cc26c8eca68)
+++ b/ee/lib/gitlab/llm/open_ai/completions/explain_vulnerability.rb	(date 1682391356329)
@@ -10,6 +10,7 @@
           end
 
           def execute(user, vulnerability, _options)
+            raise "Custom error message"
             template = template_class.new(vulnerability)
             response = response_for(user, template)
ksnip_20230424-165635

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #408343 (closed)

Edited by Daniel Tian

Merge request reports