The local cache quota is actually a CAS cache quota

Summary

To set a local cache quota we need to specify something like this in our user configuration (typically at ~/.config/buildstream.conf):

cache:
  quota: 100G

Now, having read our documentation regarding the cache quota, one would expect that this is a quota for the entire local cache, that is (typically) located at ~/.cache/buildstream. However, it seems to me that our quota is actually a quota on ~/.cache/buildstream/cas, the cas directory of the local cache.

Discovery

  • To discover this, I have two stack elements, which, when built separately, each leave ~/.cache/buildsteam at 10G.
  • I built the first stack element: du -sh ~/.cache/buildstream returns 11G
  • I then set a cache quota to 18G
  • I then built the second stack element (expecting a cleanup job), du -sh ~/.cache/buildstream returns 20G. And there was no cleanup job.
  • du -sh ~/.cache/buildstream/cas returns 16G.

From looking at our CASQuota class, it seems like the quota is actually for the local CAS cache, not the local cache.

In my opinion, this is a bug, naively, one would expect that a cache quota means that your entire cache should not exceed this quota. If it is intentional, then we need to make this clear in the documentation. If not, we need to ensure that the quota is applied to the entire local cache.