Suggestion for better artifact handling
Description
I have a proposal for an improved way of handling artifacts in what i feel is the most common use case in our project.
Proposal
Today I can say how long the artifacts should be stored. But what happens when you do this only as a way to transport your artifacts to the next step in the pipeline, but don't need the artifacts when the pipeline is done?
You're left with two options as i see it; either you set a very long time and waste space to make sure the artifacts are there for the entire pipelines life or you set a very short time and pray that the following steps start within that time. One reason it might take a log time to get through the entire pipeline could be that there is a shortage of runners at the moment, due to for example peak hours.
My proposal is simply this. Introduce a "Pipeline time/scope" for the artifacts. So as long as the pipeline is running, keep the artifacts. As soon as the pipeline is complete, destroy the artifacts. This way you keep the artifacts for as long as they are needed, but not longer.
I feel the current way of specifying a time also has a place. This proposal doesn't suggest removing the current behavior, just add more options to it.