Allow configurable timeout for cloud-hosted AI Gateway requests

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Release notes

Allow self-managed GitLab instances using cloud-hosted Duo to configure the timeout for AI Gateway requests, similar to the existing functionality for self-hosted models.

Problem to solve

Self-managed GitLab instances connecting to the cloud AI Gateway (cloud.gitlab.com) may experience timeout errors (A9999, A1000) when AI responses take longer than the default 60-second timeout to complete.

This can occur when:

  • A proxy server between the GitLab instance and cloud.gitlab.com adds latency (e.g., due to security scanning)
  • Complex or lengthy prompts require more processing time
  • Network conditions introduce delays

Currently, the configurable AI Gateway timeout feature introduced in 18.7 (see issue 567878) only applies to self-hosted models. Customers using cloud-hosted Duo have no way to extend the timeout, even when their infrastructure legitimately requires more time.

When timeouts occur mid-stream, responses can be truncated, which may trigger secondary errors in the response parser (e.g., the undefined method 'start_with?' for nil:NilClass error in final_answer_parser.rb - see issue 543173).

Proposal

Extend the existing AI Gateway timeout configuration (Admin > GitLab Duo > AI Gateway request timeout) to also apply to requests made to the cloud-hosted AI Gateway (cloud.gitlab.com), not just self-hosted models.

This would allow self-managed customers with slower network paths to cloud.gitlab.com to configure an appropriate timeout value (60-600 seconds) that accounts for their infrastructure constraints.

Intended users

  • Self-managed GitLab administrators whose instances connect to the cloud AI Gateway through proxies or slower network paths
  • Enterprise customers with security infrastructure that adds latency to outbound requests

Feature Usage Metrics

  • Number of A1000/A9999 timeout-related errors for cloud AI Gateway requests
  • Adoption rate of custom timeout configuration once available

Does this feature require an audit event?

No

Edited by 🤖 GitLab Bot 🤖