project.conf configuration parameters should be part of the cache calculation

Background

This is a problem because you can change a configuration parameter in project.conf, but it doesn't actually take effect until all the elements of your project get rebuild at some point in the future

For example, we have changed a configuration parameter in freedesktop-sdk/freedesktop-sdk!317 (merged), and we got it merged thinking everything was correct (rebuild against that new configuration)

But we had to revert it at freedesktop-sdk/freedesktop-sdk!325 (merged) when we noticed that the system didn't actually got rebuild

Acceptance Criteria

To avoid confusion, I'd say project.conf should be part of the cache-key calculation

changed the description

I'm not sure about this specific case, but it reminds me that I believe BST needs versioning of its cache-key algorithm, similar to YBD. And future versions of BST should be able to calculate using old versions of the algorithm.

@tristanvb i fear you have previously disagreed with the above, and take the view that anyone wanting old cache-key algorithms needs to use old version of BST... is that correct?

I agree that changing fail-on-overlap should be part of the cache key calculation and thus trigger a rebuild. However, we shouldn't blindly add all project.conf settings to the cache key. E.g., the artifact server setting shouldn't affect the cache key.

I'm not sure about this specific case, but it reminds me that I believe BST needs versioning of its cache-key algorithm, similar to YBD. And future versions of BST should be able to calculate using old versions of the algorithm.

We have a core artifact version number that we increment when we change the cache key algorithm or the artifact structure in an incompatible way. A particular version of BuildStream indeed only supports a single artifact version.

As far as I can tell, this is completely orthogonal to the issue raised here, though. Can you please open a separate issue for that where this can be discussed in more detail?

mentioned in issue #474 (closed)

It's somewhat orthogonal, but not completely, imo.

If this issue is resolved by changing the algorithm, without providing support for previous versions, it effectively forces a flag-day on users afaict.

mentioned in commit 54de7d55

mentioned in merge request !543 (merged)

If this issue is resolved by changing the algorithm, without providing support for previous versions, it effectively forces a flag-day on users afaict.

Side note, the fix for this changes the cache key only for projects that enable fail-on-overlap, not for all projects.

mentioned in commit 99fd1584

mentioned in commit f770ab1c

closed via merge request !543 (merged)