BuildStream should allow us to disable pushing for a build

Once we’ve declared an artifact server (with push/pull authentication) in our project/user.conf, BuildStream will remotely cache by default. So, each time we bst build something, BuildStream will try to push each element/artifact in the pipeline, one by one.

I think that it’d be great if we had an option where we could disable pushing for bst build and I’ll explain why below.

For large projects, caching is expensive, and I want to try to avoid remotely caching things that I don’t need to.

Argument 1 - I don’t want to push to my cache until I’m happy everything is correct.

I think the title of this argument clearly explains the intent.

Argument 2 - I don’t want to cache my junctioned dependencies as they’re already cached elsewhere.

I have a project which heavily depends on many elements in a junction. These junctioned elements are already cached in the junction’s remote cache. However, when I build my project, artifacts from the junction’s remote cache will be pulled down, used in my local build and then pushed to my project’s remote cache. So the same artifact has been cached twice, in two separate caches.

My remote cache is finite and the remote expiry mechanisms we have are not yet guaranteed to work / clean quickly. And in addition to this, my project’s cache is growing at an extremely fast rate (on the order of tens of gigabytes per day).

Because of this, I only want to populate this project’s remote cache with this project’s elements - not the junction’s elements. In order to do this, I have to build elements with a project.conf that only has pull authentication, then amend the project.conf so we can push, and then bst push <the project’s elements>. Again, wouldn’t it be nice if we could just disable pushing on a build so that I wouldn’t have to faff about with swapping in and out project.confs

Extension to the idea

Perhaps we could take this further and be more granular and have an option to say “Don’t push my junctioned elements”, but I’m unsure whether this use-case would benefit as many, and I think that the “don’t push during this build” is definitely a prerequisite to this.