Performance issues with running the artifact cache

This is an issues raised from the freedesktop-sdk project.

We are currently running the Buildstream artifact cache on a public Digital ocean droplet with specs:

• 1 CPU
• 8GB RAM
• 5TB Bandwith
• Running Fedora 27

We use this to cache artifacts built by out Gitlab CI runners, so these artifacts can be shared.

However when we run anything over 4 builds (4 Gitlab runners), we are seeing the servers CPU at 100% usage, causing future builds to fail.

We know the current hardware is not very powerful (we are in the process of replacing it with a better one), but we feel this is something that could be further investigated, in case there were a performance issue with the bst cache server.