Support separate end points for artifact caches

Description

This MR allows specifying a split artifact cache, as follows:

artifacts:
  - url: https://localhost:11002
    server-cert: temp_keys/server.crt
    client-key: temp_keys/client.key
    client-cert: temp_keys/client.crt
    push: true
    type: index
  - url: https://localhost:11004
    server-cert: temp_keys/server.crt
    client-key: temp_keys/client.key
    client-cert: temp_keys/client.crt
    push: true
    type: storage

This will effect that buildstream sends artifact protos to the index server, and artifact cas-blobs to the storage server.

The branch currently allows successfully pushing/pulling artifacts and sources with a split server. It's not fully tested yet.

There are still a number of things missing:

• Split off bst source push/pull into a different MR
• Give a better error message when we discover that blobs are missing during an artifact proto push.
  • We should use FindMissingBlobs to make sure that not everything's missing, we don't want misleading errors :)
  • In the future, we'll want to include artifact size in protos, so that we can say "this artifact exceeds cache limits"
• Clear out FIXMEs
  • In particular, we should continue if we receive an error from a specific artifact cache.
• Clean up comments
• Write documentation

If you want to manually try this branch you can use the following docker-compose.yml file to run two artifact services locally, and then access them using the previous project.conf settings:

Note that you will need to clone buildstream/buildstream-docker-images, checkout tlater/cache-endpoints, and place this in the artifact-cache directory. You will also likely need to change the username in the volume path :)

version: '3'
services:
  artifacts:
    build:
      context: .
      args:
        BRANCH: tlater/cache-endpoints
    image: buildstream/artifact-cache:local
    ports:
      - "11001:11001"
      - "11002:11002"
    volumes:
      - /home/tlater/Scratch/temp_keys/:/home/artifacts/keys:z
    command: "bst-artifact-cache-start"

  cas:
    build:
      context: .
      args:
        BRANCH: tlater/cache-endpoints
    image: buildstream/artifact-cache:local
    ports:
      - "11003:11001"
      - "11004:11002"
    volumes:
      - /home/tlater/Scratch/temp_keys/:/home/artifacts/keys:z
    command: "bst-artifact-cache-start"

This merge request, when approved, will close: #1041 (closed)

---