Fix AI Gateway GCP credentials error
AI Gateway currently fails to start up on EKS with the following error. This error is caused by missing Application Default Credentials.
Possible Solutions
-
Disable Cloud Profiler
Disable the Cloud Profiler by setting the
AIGW_GOOGLE_CLOUD_PROFILER__ENABLEDenvironment variable tofalse. -
Create a Cloud Profiler Service Account
Create a GCP service account and export the credentials JSON to Vault as a "workload secret". With a small change to the Helm charts, we can mount the workload secret as a volume. Last, set the
GOOGLE_APPLICATION_CREDENTIALSenvironment variable to point to the location of the secret.
Error Message
Traceback (most recent call last):
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/bin/ai_gateway", line 6, in <module>
sys.exit(run_app())
^^^^^^^^^
File "/home/aigateway/app/ai_gateway/main.py", line 10, in run_app
uvicorn.run(
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/main.py", line 580, in run
server.run()
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/server.py", line 67, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/runners.py", line 195, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/server.py", line 71, in serve
await self._serve(sockets)
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/server.py", line 78, in _serve
config.load()
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/config.py", line 442, in load
self.loaded_app = self.loaded_app()
^^^^^^^^^^^^^^^^^
File "/home/aigateway/app/ai_gateway/app.py", line 28, in get_app
app = create_fast_api_server(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/aigateway/app/ai_gateway/api/server.py", line 166, in create_fast_api_server
setup_profiling(config.google_cloud_profiler)
File "/home/aigateway/app/ai_gateway/profiling.py", line 16, in setup_profiling
googlecloudprofiler.start(
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/googlecloudprofiler/__init__.py", line 125, in start
project_id = profiler_client.setup_auth(project_id, service_account_json_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/googlecloudprofiler/client.py", line 129, in setup_auth
self._credentials, credentials_project_id = google.auth.default(
^^^^^^^^^^^^^^^^^^^^
File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/google/auth/_default.py", line 685, in default
raise exceptions.DefaultCredentialsError(_CLOUD_SDK_MISSING_CREDENTIALS)
google.auth.exceptions.DefaultCredentialsError: Your default credentials were not found. To set up Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information.
Edited by Florian Forster