Make the bst-artifact-server proxy CAS storage calls to buildbox-casd

Background

We recently discussed the question of scalability in bst-artifact-server on the mailing list, where we decided that it would make sense to allow the server to proxy out RPCs to other CAS servers so that we can use existing, more efficient backends to handle our artifacts for groups with heavier workloads.

!1540 (merged) introduced similar client-oriented behavior that makes this less important for scaling reasons, but proxying to buildbox-casd is still useful to help clean up this code and reduce the reliance on BuildStream internals, which should make both parts of the codebase more maintainable and perhaps allow future splitting of the project. This is also a remaining part of the work for #1042.

Task description

We should change buildbox-casd so that it no longer relies on BuildStream's CASCache for its CAS-related RPC calls, but instead proxies these to buildbox-casd.

This could be done as such:

  • Start launching buildbox-casd as part of bst-artifact-server's initialization
    • While technically obsoleted as part of !1540 (merged), keeping this compartmentalized would be nice, so that proxying could be later implemented easily.
  • Forward CAS-related RPC calls to buildbox-casd

Acceptance Criteria

bst-artifact-server no longer makes calls to CAS via BuildStream's internal abstractions.