Customers should have confidence in their Cloud Connector configuration

Context

We've been receiving a number of requests from customers who either ran into problems with their network proxy/firewall setup or simply wanted to verify that it is correctly set up (sometimes before actually buying Duo Pro.) The network settings we require are documented here.

Vision

Customers should have confidence that their system is set up for AI-related use cases and ready-to-go.

Open Questions

  • Where should customers receive this feedback?
    • ie, via a Rake task, somewhere in the GitLab UX, or somewhere else entirely?
  • How should this feedback fit into the customer's overall purchasing and onboarding experience to GitLab Duo?
  • Should this be part of a generalizable UX pattern for other GitLab-related network connections? (linked)

Customer Pain Points

  • It's unclear whether the set up is correctly done.
  • Customers that use GitLab behind proxies are exposed to low-level configuration details such which of our systems must be able to dial cloud.gitlab.com.

Potential Solutions

Configuration simplification

Instead of pointing customers to deployment-specific templates that require tweaking several env vars for different systems (which are an implementation detail to CC and AI), we could provide a simpler experience, such as instead of asking "set HTTP_PROXY and HTTPS_PROXY for Rails and Workhorse", we ask to set WEB_PROXY_URL once and then map this automatically to either HTTP_PROXY and/or HTTPS_PROXY for all services that need to read these (Rails, Workhorse).

Connection verification

We should consider providing a tool customers can use to verify that all necessary connections work as intended, e.g.:

  • Puma/Workhorse HTTP to cloud.gitlab.com (Code Suggestions requests)
  • Sidekiq HTTP to cloud.gitlab.com (Duo Chat requests)
  • Sidekiq HTTP to customers.gitlab.com (CC access data sync)

This tool could be a Rake task they invoke or shown on a page in the GitLab UI. A stretch goal could be to provide an actual endpoint that returns a fake code suggestion, even when no JWT is present (otherwise all of these will just 404 but for connectivity validation that is enough.)

Token visibility

There has been at least one case where a customer had everything set up, but just required an extra click to sync the licence, thus get a new access token. We could help here by providing some visibility on whether an access token has been correctly synced and is valid.

Related Problems

  • There are complex workflows for AI settings, privacy controls, and the banning of AI features across organizations / projects / groups. How do we help users and administrators understand the difference between something doesn't work because of:
    • User/administrator error (in setup steps),
    • Configuration error (mis-applied settings), or
    • Lack of access (everything is setup, but somebody which higher permissions has denied access).

Admin Journey

1. Network setup

The admin checks our guidance for network set up https://docs.gitlab.com/ee/user/ai_features_enable.html#configure-gitlab-duo-on-a-self-managed-instance. This has two sections:

  • Allow outbound connections from GitLab instance

This is where the admin must configure their firewall/proxy (if they have one) to allow outbound connections to:

cloud.gitlab.com and customers.gitlab.com

If they have a proxy server, they also need to update Gitlab config files to ensure the outbound connections to cloud.gitlab.com use this proxy

Pain Point 1.1: Once the admin has configured their firewall and/or proxy, they have no way to check that the connection from GitLab server instance actually works.

Pain Point 1.2: Some customers have firewalls/proxies that buffer responses from GitLab's AI services. This means that streamed responses from the LLM's get buffered and returned to the client only once the response is complete. This increases the perceived latency of the service.

  • Allow inbound connections from clients to the GitLab instance

In the cases where the customers are operating in a stricter networking environment, it is possible that the admin needs to allow the client IDE's etc to be able to access the GitLab server on https and also websockets

Pain Point 1.3 (? need to validate): There is no easy way in the IDE plugins to validate that this connection is working

2. Purchase GitLab Duo

The admin purchases GitLab Duo from customers.gitlab.com (at the moment this is a sales led process). Once complete, they expect this to then work in their GitLab instance. There is a token sync process that runs once a day at the same time as the Cloud Licence sync. If the customer waits until then everything will be fine, but if they want to set up immediately they must go to the Licence Management page and refresh the licence. This will enable the seat assignment functionality so the admin can assign users to be able to use GitLab Duo.

Pain Point 2.1: Once GitLab Duo is purchase there is no visibility on whether a valid token has been synced that can be used to communicate with GitLab's AI services. It should be present but if there is some sort of error when getting the token the admin has to resort to checking log files to see why.

3. Test GitLab Duo from the IDE

This is where the admin experience would start to overlap with the user facing experience. Guidance on setting up Code Suggestions is here

Pain Point 3.1 If the IDE does not work, the troubleshooting procedure is quite manual and involves turning on logging and checking the logs to see what the errors might be.

Paint Point 3.2 If the error turns out to be related to a 401 error, the guidance is to resync the licence. However as described in Pain Point 2.1, there is no easy way to know from the UI if the resync has worked and a valid token has been synced.

Edited by Roger Woo