Backups take a long time if gitaly and task-runner are scheduled to the same host
Summary
We use the gitlab backup script to backup gitlab. Both gitaly and task-runner use lots of CPU in the process of backing up. If the task-runner pod is scheduled on the same host as gitaly the pods both try to use the same resource causing the backup to take significantly longer
Steps to reproduce
With gitaly and the task-runner on the same host run the backup script from the task runner and time the run
Now separate gitaly and task-runner and run the same backup script and time the run, you should see this second run is significantly faster
Current behavior
Gitaly and task-runner pods have nothing to prefer them not to be scheduled to the same host
Prefered behavior
The scheduler prefers not to schedule Gitaly and task-runner on the same host
Versions
- Chart: 2.4.6
- Platform:
- Cloud: GKE
- Kubernetes: (
kubectl version
)- Client: v1.14.0
- Server: v1.14.7-gke.14
- Helm: (
helm version
)- Client: v2.11.0
- Server: v2.11.0