Skip to content

Fix AI Gateway GCP credentials error

AI Gateway currently fails to start up on EKS with the following error. This error is caused by missing Application Default Credentials.

Possible Solutions

  • Disable Cloud Profiler

    Disable the Cloud Profiler by setting the AIGW_GOOGLE_CLOUD_PROFILER__ENABLED environment variable to false.

  • Create a Cloud Profiler Service Account

    Create a GCP service account and export the credentials JSON to Vault as a "workload secret". With a small change to the Helm charts, we can mount the workload secret as a volume. Last, set the GOOGLE_APPLICATION_CREDENTIALS environment variable to point to the location of the secret.

Error Message

Traceback (most recent call last):
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/bin/ai_gateway", line 6, in <module>
    sys.exit(run_app())
             ^^^^^^^^^
  File "/home/aigateway/app/ai_gateway/main.py", line 10, in run_app
    uvicorn.run(
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/main.py", line 580, in run
    server.run()
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/server.py", line 67, in run
    return asyncio.run(self.serve(sockets=sockets))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/server.py", line 71, in serve
    await self._serve(sockets)
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/server.py", line 78, in _serve
    config.load()
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/uvicorn/config.py", line 442, in load
    self.loaded_app = self.loaded_app()
                      ^^^^^^^^^^^^^^^^^
  File "/home/aigateway/app/ai_gateway/app.py", line 28, in get_app
    app = create_fast_api_server(config)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aigateway/app/ai_gateway/api/server.py", line 166, in create_fast_api_server
    setup_profiling(config.google_cloud_profiler)
  File "/home/aigateway/app/ai_gateway/profiling.py", line 16, in setup_profiling
    googlecloudprofiler.start(
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/googlecloudprofiler/__init__.py", line 125, in start
    project_id = profiler_client.setup_auth(project_id, service_account_json_file)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/googlecloudprofiler/client.py", line 129, in setup_auth
    self._credentials, credentials_project_id = google.auth.default(
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/home/aigateway/app/venv/ai-gateway-M6hW6iiC-py3.12/lib/python3.12/site-packages/google/auth/_default.py", line 685, in default
    raise exceptions.DefaultCredentialsError(_CLOUD_SDK_MISSING_CREDENTIALS)
google.auth.exceptions.DefaultCredentialsError: Your default credentials were not found. To set up Application Default Credentials, see https://cloud.google.com/docs/authentication/external/set-up-adc for more information.
Edited by Florian Forster