How should Shared Workspaces work?
Problem Statement
A very common use case for pipelines is to implement a series of jobs that perform a transformation and/or analysis on a working set of files. Most builds work this way essentially, but these are becoming even more common as a paradigm for data science models. At the moment GitLab provides artifacts as one option, but these are more meant for longer term storage so persisting these for a long time is a waste. We also provide caching, but the cache is best-effort (so not guaranteed) and can cause surprises if its used as a workspace by causing builds to unexpectedly fail or behave strangely because they are missing their intermediary files.
Additional details at #29265