Fix errors returned as 200 HTTP status code (Chat)

What does this merge request do and why?

In gitlab-org/gitlab#433213 (comment 1698278046) and https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/381+, we've found out that the server error is returned with 200 status. This is hard for clients to figure out why the server failed to process the request, and hard for us to investigate the error requests because they're buried in 200.

This MR fixes the issue for Chat endpoint by defining the status code, which follows FastAPI error handling guidance.

Related to Improve observability of Duo Chat with v1/agent... (#371 - closed) and Handle exceptions from AI Gateway (gitlab-org/gitlab#434927 - closed)

How to set up and validate locally

Inject a dummy error:

    try:
        raise AnthropicAPIConnectionError("dummy")
    
        completion = await model.generate(
            prefix=payload.content,
            _suffix="",
            stream=chat_request.stream,
        )

Request to the v1/chat/agent endpoint:

curl -X POST -H "Content-Type: application/json" -d '{
  "prompt_components": [
    {
      "type": "string",
      "metadata": {
        "source": "string",
        "version": "string"
      },
      "payload": {
        "content": "\n\nHuman: Hi, How are you?\n\nAssistant:",
        "provider": "anthropic",
        "model": "claude-2.0",
        "params": {
          "stop_sequences": [
            "\n\nHuman",
            "Observation:"
          ],
          "temperature": 0.2,
          "max_tokens_to_sample": 2048
        }
      }
    }
  ],
  "stream": false
}' http://0.0.0.0:5052/v1/chat/agent

Before:

{
    "url": "http://0.0.0.0:5052/v1/chat/agent",
    "path": "/v1/chat/agent",
    "status_code": 200,
    "method": "POST",
    "correlation_id": "8d97ade1f6184e8f834c60b0e1a2d5c0",
    "http_version": "1.1",
    "client_ip": "127.0.0.1",
    "client_port": 51728,
    "duration_s": 0.8521455140016769,
    "cpu_s": 0.06053873600000026,
    "user_agent": "curl/8.4.0",
    "gitlab_instance_id": null,
    "gitlab_global_user_id": null,
    "gitlab_host_name": null,
    "gitlab_saas_namespace_ids": null,
    "gitlab_realm": null,
    "logger": "api.access",
    "level": "info",
    "type": "mlops",
    "stage": "main",
    "timestamp": "2023-12-18T05:32:31.423184Z",
    "message": "127.0.0.1:51728 - \"POST /v1/chat/agent HTTP/1.1\" 200"
}

Response:

{"response":"","metadata":{"provider":"anthropic","model":"claude-2.0","timestamp":1702877551}}

Notice that two bugs here:

• An error happened in the server, however, status_code is returned as 200.
• Response is returned as empty string, which is confusing as if that's the output from the Anthropic.

After:

{
    "url": "http://0.0.0.0:5052/v1/chat/agent",
    "path": "/v1/chat/agent",
    "status_code": 502,
    "method": "POST",
    "correlation_id": "690d342ca66b407f8e3eb4b93888443f",
    "http_version": "1.1",
    "client_ip": "127.0.0.1",
    "client_port": 46334,
    "duration_s": 0.039347043999441667,
    "cpu_s": 0.03925611999999967,
    "user_agent": "curl/8.4.0",
    "gitlab_instance_id": null,
    "gitlab_global_user_id": null,
    "gitlab_host_name": null,
    "gitlab_saas_namespace_ids": null,
    "gitlab_realm": null,
    "meta.feature_category": "duo_chat",
    "logger": "api.access",
    "level": "info",
    "type": "mlops",
    "stage": "main",
    "timestamp": "2023-12-28T10:24:29.208107Z",
    "message": "127.0.0.1:46334 - \"POST /v1/chat/agent HTTP/1.1\" 502"
}

Response:

{"detail":"Anthropic API Connection Error."}

Error log (This is similar to GitLab-Rails's ErrorTracking module):

{
    "status_code": null,
    "backtrace": "Traceback (most recent call last):\n  File \"/home/shinya/ai-assist/ai_gateway/api/v1/chat/agent.py\", line 112, in chat\n    raise AnthropicAPIConnectionError(\"dummy\")\nai_gateway.models.anthropic.AnthropicAPIConnectionError: dummy\n",
    "correlation_id": "690d342ca66b407f8e3eb4b93888443f",
    "logger": "exceptions",
    "level": "error",
    "type": "mlops",
    "stage": "main",
    "timestamp": "2023-12-28T10:24:29.207680Z",
    "message": "dummy"
}

Merge request checklist

• Tests added for new functionality. If not, please raise an issue to follow up.
• Documentation added/updated, if needed.