-
- Downloads
Revamp calculation of cache size code
Before this commit, the cache size was being calculated every time an artifact was committed to or pulled to the artifact cache. This is a very IO demanding operation that doesn't scale as the cache gets bigger. Now that the commit() and pull() methods of the ArtifactCache return the number of bytes added, we can rework the code to calculate the size of The cache only once at startup and to dynamically update it afterwards. The ArtifactClean.clean() is also changed to return the number of bytes removed instead of the new size of the cache, so that we no longer need the ArtifactClean.clean._set_cache_size() method. The CacheSize job is no longer used and is removed. Fixes #573.
Showing
- buildstream/_artifactcache/artifactcache.py 26 additions, 56 deletionsbuildstream/_artifactcache/artifactcache.py
- buildstream/_artifactcache/cascache.py 1 addition, 7 deletionsbuildstream/_artifactcache/cascache.py
- buildstream/_frontend/app.py 4 additions, 0 deletionsbuildstream/_frontend/app.py
- buildstream/_scheduler/jobs/__init__.py 0 additions, 1 deletionbuildstream/_scheduler/jobs/__init__.py
- buildstream/_scheduler/jobs/cachesizejob.py 0 additions, 38 deletionsbuildstream/_scheduler/jobs/cachesizejob.py
- buildstream/_scheduler/jobs/cleanupjob.py 6 additions, 5 deletionsbuildstream/_scheduler/jobs/cleanupjob.py
- buildstream/_scheduler/jobs/elementjob.py 0 additions, 2 deletionsbuildstream/_scheduler/jobs/elementjob.py
- buildstream/_scheduler/queues/buildqueue.py 0 additions, 13 deletionsbuildstream/_scheduler/queues/buildqueue.py
- buildstream/_scheduler/queues/pullqueue.py 2 additions, 5 deletionsbuildstream/_scheduler/queues/pullqueue.py
- buildstream/_scheduler/queues/queue.py 13 additions, 2 deletionsbuildstream/_scheduler/queues/queue.py
- buildstream/_scheduler/scheduler.py 4 additions, 16 deletionsbuildstream/_scheduler/scheduler.py
Loading
Please register or sign in to comment