Buildstream needlessly stages junctions into tmpdir
Summary
Buildstream invocations are slower than they need to be - Part of junction loading logic involves creating a temporary directory and staging files into it. At the very least, we oughtn't need to stage the junction for every buildstream invocation.
Steps to reproduce
tests/format/jnunctions.py::test_git_show will exercise the appropriate part of the code (buildstream/_loader/loader.py:552)
The example in doc/examples/junctions is not sufficient, as there is a special case for "local" sources.
What is the current bug behavior?
The source is staged on every buildstream invocation.
What is the expected correct behavior?
If possible, the source is not staged at all, the files being read directly from the source cache.
Since we have no guarantees that sources in sourcedir are accessible as individual files, this would have to wait for an implementation of a case-based source cache (https://mail.gnome.org/archives/buildstream-list/2019-February/msg00000.html).
A decent fall-back would be to not have to stage the source every buildstream invocation.
Possible fixes
- Instead of staging into a temporary directory, stage into the directory
$junction_element_name/$junction_element_cache_key. - If the cache-keyed directory already exists, use that.
- If the cache-keyed directory doesn't exist, but there are other directories, delete them and stage as in 1.
- Investigate whether loader.py::cleanup is still useful.
Fortunately, _stage_sources_at already guarantees atomicity, so aborting buildstream at an inopportune time won't leave a corrupted source checkout.