Add auth token options to remote storage and ac

Before raising this MR, consider whether the following are required, and complete if so:

• Unit tests
• Metrics
• Documentation update(s)

If not required, please explain in brief why not.

Description

If a remote storage/remote action cache option is used as the storage or the action cache, it is possible that the remote resource requires it's clients to be authenticated. There is no mechanism to add a authentication token yet for the remote source.

This MR aims to provide a mechanism that a auth token can be specified in the Buildgrid for the remote service. One thing to note is about authorization.

If we have the following case, there is a execution service using a remote CAS and an remote AC, both of which require authorization

• The storage and ac contain an auth token representing the user buildgrid
• Identity x authorizes with the execution service
• The execution service will not use the identity x but rather use the token user buildgrid

This behavior is indented, as the execution service should always be able to access the CAS and the Storage irrespective of the identify of the client calling the service

Changes proposed in this merge request:

• Add a authorization-token config to the remote storage and remote action cache

Validation

Testing this proved to be difficult but here is the steps I took. All files are

Test files for https://gitlab.com/BuildGrid/buildgrid/-/merge_requests/1027 ($3687131) · Snippets · Snippets · GitLab

• Generated a pair of certs and stored it at an appropriate place

• Started an ha proxy for tls termination

  • /sbin/haproxy -f ha-proxy.conf

• Started a jwks server

  • python3 jwt_server.py
  • I used a virtual environment and installed the required packages

• Started the first buildgrid server

  • bgd server start with_auth.yml -vvv

• Started the second buildgrid server

  • bgd server start with_remote_auth.yml -vvv

• Set GRPC_DEFAULT_SSL_ROOTS_FILE_PATH to the appropriate public cert location you generated in the first step

• Get an auth token from the jwt server

  curl http://127.0.0.1:5001/get-jwt/buildgrid-dev > test/temp.token

• Start a worker *

  buildbox-worker --bots-remote=https://localhost:50052 --cas-remote=https://localhost:60052 \
      --buildbox-run=buildbox-run-hosttools --bots-access-token "test/temp.token" \
      --cas-access-token "test/temp.token" --runner-arg=--disable-localcas my_bot

• Run trexe *

  trexe --exec-remote "https://localhost:50052" --exec-instance "$INSTANCE_NAME" \
              --cas-remote "https://localhost:60052" --cas-instance "$INSTANCE_NAME" \
              --ac-remote "https://localhost:60052" --ac-instance "$INSTANCE_NAME" \
              --access-token "test/temp.token" \
              --input-path "test" \
              --wait /bin/echo "hi-$INSTANCE_NAME"

  Exit code should be zero