Skip to content

feat: add readiness endpoint

Bob Van Landuyt requested to merge bvl/startup-probe into main

feat: add a readiness endpoint

This adds a readiness endpoint that can be polled by cloud run in order to determine if the container is ready to serve traffic.

During these readiness checks we validate if we can reach Anthropic & Vertex at the specified endpoints. This avoids us accidentally deploying to a GCP region where vertex is not available. As we've seen in https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/450, the documentation is not always reliable.

Because the readiness check is a publicly accessible endpoint, we memoize the result of the checks that passed. This avoids reaching out to the model providers multiple times after the application has already started.

For #444 (closed)

Edited by Bob Van Landuyt

Merge request reports