Draft: Migrate DocsClient to new Tokens interface

What does this MR do and why?

Migrate DocsClient to the updated Token interface.

This is a refactoring and should not result in any user-facing changes.

How to test locally

Pull this branch.
Make sure you run GitLab as gitlab.com/SaaS (GITLAB_SIMULATE_SAAS=1).
Restart GitLab.
In rails console, enable the FF: Feature.enable(:documentation_search_new_tokens_path)
Open Duo Chat.
Ask Chat to search through GitLab docs.
I used this prompt to invoke that:

explain me how gitlab ci works in 3 sentences. search gitlab docs.

First of all, you should get proper response.
Second, in tail -f log/llm.log you will see relevant log records:

{
  "severity": "INFO",
  "time": "2025-08-12T09:38:38.984Z",
  "correlation_id": "01K2ETAM7S334KZ9QRM7QBZ2XE",
  "meta.caller_id": "Llm::CompletionWorker",
  "meta.feature_category": "ai_abstraction_layer",
  "meta.organization_id": 1,
  "meta.remote_ip": "127.0.0.1",
  "meta.http_router_rule_action": "classify",
  "meta.http_router_rule_type": "SESSION_PREFIX",
  "meta.user": "root",
  "meta.user_id": 1,
  "meta.client_id": "user/1",
  "meta.root_caller_id": "GraphqlController#execute",
  "options": {},
  "message": "Searching docs from AI Gateway",
  "class": "Gitlab::Llm::AiGateway::DocsClient",
  "ai_event_name": "performing_request",
  "ai_component": "duo_chat"
}

Finally, you should also see relevant logs in your AI GW (if running locally):

127.0.0.1:61533 - "POST /v2/chat/agent HTTP/1.1" 200  
127.0.0.1:61537 - "POST /v1/search/gitlab-docs HTTP/1.1" 200   
127.0.0.1:61539 - "POST /v1/prompts/chat/documentation_search HTTP/1.1" 200

---

I also debug-printed the result of TokenIssuer.new() (under if use_new_token_path_for?) -> with that I confirmed that we indeed enter the new issuing path. By inspecting the token and verifying I am getting the correct response from AI GW, I confirmed that the functionality remained unaffected ✅

The user-facing behavior will be similar for master and this branch (in any FF state) because it is a refactor.
You can also check similar steps in Self-Managed to make sure Docs Search (and Chat) works as before.

References

Related to #560757 (closed)