Finer grained source caching (tech-debt)

Background

A follow up to !1124 (merged) and !1214 (merged), sources are currently cached and staged altogether, due to some plugins (i.e. patch) requiring previous sources staged to make sense. Being able to more finely cache sources would be beneficial however, allowing more sharing between both elements and different projects.

To allow for this we need to add another flag to plugins, very similar to the existing flags BST_REQUIRES_PREVIOUS_SOURCES_TRACK/FETCH, for staging. This should only require changes to logic in the Element class, to ensure that sources are staged appropriately depending on whether they require previous sources or not. All sources before a source plugin marked to require previous sources to stage will have to be staged together as they currently are, all afterwards can be staged separately. This will have to be taken into account both where they are staged and where they are cached, as well as for key generation.

Task description

  • Add BST_REQUIRES_PREVIOUS_SOURCES_TRACK, and logic in Element that uses it.
  • Look through and mark relevant plugins with this flag (think it's only patch in the main repo)
  • Add some tests to ensure that sources are staged separately when possible.

Acceptance Criteria

Extra tests that show sources are staged separately when appropriate.


Edited by Raoul Hidalgo Charman