Description

This optimizes how BuildStream handles states, and reduces the number of calls that are necessary to detect whether something is cached or not.

Changes proposed in this merge request:

• add a is_resolved on sources to determine whether the source is resolved or not. Default implementation is self.get_ref() is not None
• add a is_cached() on sources to determine whether the source has a local copy of its data
• Remove get_consistency() on Source, which only allows coarse-grained state tracking and is not needed by 1 and 2.
• separate 'in source cache' and 'cached by plugin' logic in Element

Fixes

Fix #1126 (closed)

BREAKING BEHAVIORS

This optimization ensures that we do not call the check for a plugin of whether its sources are cached or not.

This means that if sources are in the internal source cache, we don't call the validate_cache part of the plugin anymore, which means that warnings will not be shown/will not error the build as soon as the source is in the source cache.

Since this is a breaking behavior it will need to be synchronized with

Benchmarks

Using the dummy debian project

• Finished in: 84 mins, 32 secs
• Number of builders: [4, 8, 12, 24]
• Target files: base-files/base-files.bst
• Number of runs: 4
• Number of warmups: 2
• Python versions: py37

There were 4 runs of each command.

Using a real internal project