Support separate end points for artifact caches
Background
BuildStream currently assumes that the ArtifactService/ReferenceService endpoint and the CAS endpoint are the same for a cache, this is not the case if we're using other projects for remote CAS (i.e. buildgrid). This also means that the storage service for remote execution can be the same as for buildstream caches.
Task description
-
Extend configuration to support separate end points for the artifact service and the CAS. Should also be able to specify that both are provide by one end point with previous, which should also mean old configurations don't break. -
CasRemote
/ArtifactRemote
will need some refactoring to support initialising these separately in different remote objects. -
Update ArtifactCache to look at these separately, querying the artifact end points to get the metadata and then the CAS end point to pull/push artifact files. -
refactoring some code to separate it into artifact push/pull methods and then storage push_pull methods. -
Loop through services and then storage end points to pull/push
-
-
Reference/source cache methods will similarly have to change to support separate reference/source service and storage service. -
Add some tests for this. -
Update docs.
Acceptance Criteria
Able to use a separate ArtifactService
and CASCache
to cache artifacts.
Edited by Javier Jardón