Improve observability of Duo Chat with v1/agent/chat endpoint

Problem to solve

Code Generation and Suggestions are instrumented by the TextGenModelInstrumentator, however, Duo Chat v1/agent/chat endpoint is not instrumented yet.

This is important to measure key metrics like prompt_length and prompt_length_stripped otherwise we have no observability on the endpoint.

Technically speaking, this is instrumented in GitLab-Rails gitlab-org/gitlab!138274 (diffs). However, it should be tracked in AI Gateway instead.

Proposal

• We need to set the X-Request-ID to the header ee/lib/gitlab/llm/ai_gateway/client.rb. Currently, Duo Chat on Rails and AI Gateway have different correlation IDs.

Further details

Links / references