- This is behind a feature flag for testing
- Uses request body format needed for messages API, which is now
  supported by the AI Gateway:
  gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!668
- #444629

Nicolas Dular authored 1 year ago and Michał Wielich committed 1 year ago

This adds the client information to our AI action tracking by passing
forward the user agent and parsing it. For now we only distinguish
between vscode and web.

This model will contain related state of the system
for specific message

Pure refactoring

This replaces the legacy openai_experimentation FF usages with
the more generic ai_global_switch ops FF

Changelog: changed
EE: true

Alexandru Croitor authored 1 year ago and Nicolas Dular committed 1 year ago

This FF is added to replace the legacy openai_experimentation

Pavel Shutsin authored 1 year ago and Nicolas Dular committed 1 year ago

It makes completion classes aware of prompt message object. 
Pure refactoring, no user facing changes.

Pavel Shutsin authored 1 year ago and Nicolas Dular committed 1 year ago

It reworks AI Response emitting to work with AiMessage model. 
No user facing changes.

Related functionality was removed so this
codepath is not used anymore

Nicolas Dular authored 1 year ago and Gosia Ksionek committed 1 year ago

Sidekiq can cause issues with code reloading in development. To overcome
this, it's now possible to set `LLM_DEVELOPMENT_SYNC_EXECUTION=1`
in development which executes AI actions synchronously.

Jan Provaznik authored 1 year ago and Bob Van Landuyt committed 1 year ago

* adds SLI for overall time needed to serve user's request (currently
  20s)
* adds histogram for overall worker duration (with service/category
  labels)

Nicolas Dular authored 1 year ago and Alexandru Croitor committed 1 year ago

Adds tracking to AI features with an approximated measurement of our
token usage for Anthropic and Vertex.

It enables us to group token usage per feature or per user.

Jan Provaznik authored 1 year ago and Gosia Ksionek committed 1 year ago

* with this change success ratio of AI requests is measured outside of
  exponential backoff
* llm_chat_answers SLI is replaced with more generic llm_completion
  which tracks error ratio of all AI actions

Nicolas Dular authored 1 year ago and Gosia Ksionek committed 1 year ago

This adds an optional client_subscription_id to the ai_completion_response
subscription. In addition to that it fixes the GraphqlTriggers to be
able to deal with optional subscription arguments.

This prepares us to allow listening only to a specific client_subscription_id
on the websocket, and to only broadcast messages based on a user_id. This
is important for the chat as well as other aiActions.

This has no breaking changes, nor changes how the subscription gets used.

Changelog: changed
EE: true

euko authored 1 year ago and Gosia Ksionek committed 1 year ago

If the current user is viewing some code blob,
the user should be able to ask the chat to
explain the code.

We will inject the blob's code into the zeroshot
executor's prompt and ask the LLMs to directly
explain the code when instructed.

To make that possible, we will make use of
the Referer header to detect if a user is viewing
a Blob. The referer url will be added as an option
to be extracted by CompletionWorker.

CompletionWorker will then attempt to resolve
and authorize the blob pointed to by the referer url.
If the blob is found and authorized,
it will be available as a context attribute, 'extra_resource'.

The zeroshot executor can then use the attribute to
include the code blob and additional prompt.

The change is guarded with a feature flag.

We fixed this before by setting `skip_cache: true` by default in
`ExecuteMethodService`. However, `SummarizeSubmittedReviewService` was
not going through the `ExecuteMethodService`.

As we only want to store messages from the `chat` action, and to fix
this for the future, the logic is now reversed and it's required to set
`cache_response: true` explicitly.

Jan Provaznik authored 1 year ago and Max Woolf committed 1 year ago

* also updates outdated product_analytics ategory

We no longer want to store and show AI messages on the chat, if not
explicitly enabled by the feature. It is now only enabled for the `chat`
AI action.
We do this by setting `skip_cache = true` by default.

It also fixes a bug where the `skip_cache` was not passed along properly
to the GraphqlSubscriptionResponseService.

Changelog: fixed
EE: true

Nicolas Dular authored 1 year ago and Igor Drozdov committed 1 year ago

Deal with optional resource_id in service and worker

Makes sure we allow using nil as resource_id in the service and the
worker.

Nicolas Dular authored 1 year ago and 🤖 GitLab Bot 🤖 committed 1 year ago

When the Agent picks the `SummarizeComments` tool, we are internally
calling the `GenerateSummaryService`, which also stores the response to
the cache and broadcasts it via the GraphQL subscription.

With `skip_cache: true` we were already able to not store the response
to the cache. This change renames the `skip_cache` option to
`internal_request` and also no longer broadcasts the response to the
client, which resulted in duplicated responses in the chat.

When the Agent picks the `SummarizeComments` tool, we are internally
calling the `GenerateSummaryService`, which also stores the response to
the cache and broadcasts it via the GraphQL subscription.

With `skip_cache: true` we were already able to not store the response
to the cache. This change renames the `skip_cache` option to
`internal_request` and also no longer broadcasts the response to the
client, which resulted in duplicated responses in the chat.

It's possible that some AI chain tools use completion service (e.g.
SummaryComments tool), in this case we want to avoid storing
request/response in cache because it's only intermediate step.

Bojan Marjanovic authored 1 year ago and George Koltsov committed 1 year ago

It removes redundant method from the code.

Nicolas Dular authored 1 year ago and Jan Provaznik committed 1 year ago

Adds a new logger and debugging statements for AI features.

Jan Provaznik authored 1 year ago and George Koltsov committed 1 year ago

* for each AI mutation it generates a unique ID
* this ID is also part of subscription message so clients can pair
  responses with original requests

- Moved the Completions::Factory to a more generic module
- Moved ExponentialBackoff to a generic AI concern

Before we make a call to AI API, we need to check if user who
actually executed the action can read the resource and if the
resource can actually be sent to AI (utilize `#send_to_ai?`).

We already have the permission check at the mutation level and
`#send_to_ai?` check in `Llm::BaseService`. But it is possible
that those permissions changes after the job is enqueued.

No changelog since this is still behind a feature flag.

Alexandru Croitor authored 1 year ago and Bojan Marjanovic committed 1 year ago

This adds backend services and sidekiq job to handle
ai completion requests to OpenAI API with potential to
extend it beyond OpenAI