Skip to content

Fix flaky test in code_suggestions_client_spec.rb

Leaminn Ma requested to merge code-suggestions-client-fix-flaky-test into master

What does this MR do and why?

Currently the tests in ee/lib/gitlab/llm/ai_gateway/code_suggestions_client.rb occasionally fail with the following error:

     WebMock::NetConnectNotAllowedError:
       Real HTTP connections are disabled. Unregistered request: POST https://cloud.gitlab.com/auth/v1/code/user_access_token with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Authorization'=>'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJqdGkiOiI5Yjk3MjU0Ni0zOWVjLTQ0YTMtYTBlNy01OTg0Y2RjODIwMzYiLCJhdWQiOiJnaXRsYWItYWktZ2F0ZXdheSIsInN1YiI6bnVsbCwiaXNzIjoiaHR0cDovL2xvY2FsaG9zdCIsImlhdCI6MTcxODIwNzc3MiwibmJmIjoxNzE4MjA3NzY3LCJleHAiOjE3MTgyMTEzNzIsImdpdGxhYl9yZWFsbSI6InNlbGYtbWFuYWdlZCIsInNjb3BlcyI6W119.MW_aQMOe3KTUkghHAqTDnRaQ-bcoJ7VP5OQhpNqadsMq8bIKisIEoIlO6QiShJs5UEbdTUFleqLqwnspx2fkfnQQPsPpQnL_hAGmp-7zieQEK44fql0C2f4BYDTdNrARhe8YTlCCnuJWUSvnZDFLgL-ARl3yursvaHOL2-DLs-3fzAJC0NqFN1OaVDskd3Leq23oVGuF_xu5nAw34CWWqHnvXMpGMfC0kYGvxrzbAgppDjdwGY9mYwCu_oEmbn1u6v_IRriOdUT5qGxwBeR1Q1VIa3kUiTRddtw6rmYp0h2V2JQXiUDfD8DPblN7OevDON-IHGXUXdmXnXbJu2T7IQ', 'Content-Type'=>'application/json', 'User-Agent'=>'Ruby', 'X-Gitlab-Authentication-Type'=>'oidc', 'X-Gitlab-Global-User-Id'=>'1l5HJGcgrJb0JDYkH3mhdnNGy0aUwb1HqikSkiCltjw=', 'X-Gitlab-Host-Name'=>'localhost', 'X-Gitlab-Instance-Id'=>'uuid-not-set', 'X-Gitlab-Realm'=>'self-managed', 'X-Gitlab-Version'=>'17.1.0', 'X-Request-Id'=>'1f93fd6d5eb6c42e6a43d3305404e84e'}

The problem is that there is a state leak between spec files that is order dependent. Further details are explained in #467246 (comment 1950115093).

The quick fix is to clear the necessary memoizations when :with_cloud_connector is used. However, it's noted in #467246 (comment 1950115093) that we should consider a more robust long term solution.

Resolves #467246 (closed) and related issues.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Before After
Screenshot_2024-06-13_at_6.01.25_PM Screenshot_2024-06-13_at_6.54.28_PM

How to set up and validate locally

  1. Before checking out this branch, you can reproduce the error by running the following:
bundle exec rspec --order defined ee/spec/requests/api/internal/ai/x_ray/scan_spec.rb:197 ee/spec/lib/gitlab/llm/ai_gateway/code_suggestions_client_spec.rb:104
  1. After checking out this branch, re-run the command above. Observe there is no error.

Related to #467246 (closed)

Edited by Leaminn Ma

Merge request reports