This commit restores the previous behavior of returning `nil` if the
response body is empty and improves the error messages around this.

Previously `Llm::AiGateway::Client#request` returned `nil` if the AI
Gateway return a `204 No Content` response. However, this makes it
impossible to discern whether the request was successful or whether
the server returned a 5xx error.

This was happening because `run_retry_with_exponential_backoff`
returned `nil` if the body were blank. To fix this, return the
HTTParty response even the body is blank, but ensure the callers
handle this.

Changelog: fixed

Refactor anthropic client, completions base and categorize questions to use new logger-related mixin

This commit makes VertexAI::Client compatible with
AI Gateway.

The change is behind the feature flag.

Instead of relying on an estimation of 4 characters being 1 token, we
now use the actual tokens we receive from the Vertex API.
In addition to that, we now track embeddings as a separate action and do
not count for it twice for input and output.

Nicolas Dular authored 1 year ago and Alexandru Croitor committed 1 year ago

Adds tracking to AI features with an approximated measurement of our
token usage for Anthropic and Vertex.

It enables us to group token usage per feature or per user.

Jan Provaznik authored 1 year ago and Gosia Ksionek committed 1 year ago

* with this change success ratio of AI requests is measured outside of
  exponential backoff
* llm_chat_answers SLI is replaced with more generic llm_completion
  which tracks error ratio of all AI actions

Alexandru Croitor authored 1 year ago and Prabakaran Murugesan committed 1 year ago

This MR adds database table vertex_gitlab_docs to store vertex
embeddigns for gitlab documentation.

This also adds the text_embeddings method to the VertexAI::Client
class to call vertex ai endpoint for building text embeddings.

re https://gitlab.com/gitlab-org/gitlab/-/issues/420939

Gregory Havenga authored 1 year ago and Kerri Miller committed 1 year ago

Changelog: changed
EE: true

Jan Provaznik authored 1 year ago and Nicolas Dular committed 1 year ago

* adds also more generic AI request SLI to monitor all provider requests

Nicolas Dular authored 1 year ago and Alexandru Croitor committed 1 year ago

Sets the log_level to INFO by default to make sure we do not log PII
data in production environments.

Changelog: changed
EE: true

Extend `Gitlab::Llm::VertexAi::Client` to:
- Add `code_completion` method to utilize `code-gecko` model

Changelog: added
EE: true

Also adding raw response to the debug log, which helps with
debugging responses from AI.

Nicolas Dular authored 1 year ago and Jan Provaznik committed 1 year ago

Adds a new logger and debugging statements for AI features.

`Gitlab::Llm::Concerns::CircuitBreaker` requires `service_name` to
be defined.

Before this change, we are only using a single `service_name` and
that means all client will use a single circuit. If a single provider
fails and the circuit opens, all providers will be affected.

To prevent that, since we have different clients (e.g. OpenAI, Vertex,
Anthropic), we define a specific service name per client.

This also includes a fix to `ExponentialBackoff` concern to raise
the correct exception to avoid a `NameError`.

Extend `Gitlab::Llm::VertexAi::Client` to:
  - Add `text` method to utilize `text-bison` model
  - Add `code` method to utilize `code-bison-001` model
  - Add `messages_chat` method to utilize multiturn `chat-bison-001` model

Changelog: added
EE: true

mo khan authored 1 year ago and Jan Provaznik committed 1 year ago

This renames the codename Tofa to Vertex AI

- Moved the Completions::Factory to a more generic module
- Moved ExponentialBackoff to a generic AI concern

Robert May authored 1 year ago and Jessie Young committed 1 year ago

Changelog: added