Split remote execution from artifact cache

Description

Currently remote execution can only specify one URL, which is used for the REAPI server (usually BuildGrid), and remote storage has to be the same as the artifact cache. This MR splits those up, so you can specify another CAS remote, and separately specify the endpoint for the execution service.

You can still specify the same server for your artifact cache and remote execution store if you want.

At the moment, local RE CAS storage is still in the same location as artifact cache storage, it's just the remotes that differ.

Proposed changes

Changes proposed in this merge request:

• Change config for remote execution so it can accept separate configuration for the execution server and storage service.
• Create our own CASCache objects in SandboxRemote instead of using the artifact cache's objects.
• Add documentation for the new config options
• Add tests for new config options
• Remove the methods in ArtifactCache which are only necessary for remote execution.

I'm not proposing to add a full CI remote execution test, but I will add some tests which check the new config option behaves sensibly.

This merge request, when approved, will close: #750 (closed)

---

doc/source/format_project.rst

@@ -231,10 +231,24 @@ using the `remote-execution` option:
   remote-execution:
 
     # A url defining a remote execution server
-    url: http://buildserver.example.com:50051
+    execution-service:
+      url: http://buildserver.example.com:50051
+    storage-service:
+    - url: https://foo.com/artifacts:11002
+      server-cert: server.crt
+      client-cert: client.crt
+      client-key: client.key
+
+The execution-server part of remote execution does not support encrypted
+connections yet, so the protocol must always be http.
+
+storage-server specifies a remote CAS store and the parameters are the
+same as those used to specify an :ref:`artifact server <artifacts>`.
 
-The url should contain a hostname and port separated by ':'. Only plain HTTP is
-currently suported (no HTTPS).
+The storage server may be the same server used for artifact
+caching. Remote execution cannot work without push access to the
+storage server, so you must specify a client certificate and key, and
+a server certificate.
 
 The Remote Execution API can be found via https://github.com/bazelbuild/remote-apis.