Skip to content

Support text response in predictions response modifier

What does this MR do and why?

When Gitlab::Llm::VertexAi::Client#text is used and the response is passed to Gitlab::Llm::VertexAi::ResponseModifiers::Predictions, #response_body will return nil. This is because it doesn't support the response format when using #text.

Example response when using #text:

{
  "predictions": [
    {
      "citationMetadata": { "citations" : [] },
      "safetyAttributes": {
        "categories": [],
        "blocked": false,
        "scores": []
      },
      "content": "content"
    }
  ]
}

This modifies the predictions response modifier to support response format.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #409509 (comment 1433293217)

Edited by Patrick Bajao

Merge request reports