SE-3098: Use multiple threads per worker, remove deployment monitoring task.
Created by: Kelketek
This pull request enables multiple threads per worker. This should solve the immediate problem of the workers being blocked without increasing memory usage or CPU in any significant sense, as all threads can share current process memory and our workloads are almost entirely IO-based. This takes us from an effective worker count of 3 to a count of 18.
To test this PR: Start up your local dev stack and try to provision a server. If the server provisions, it works. For contrast, if you were to do this on master, it would hang indefinitely as the single process would only have one thread, and a minimum of two workers is needed for provisioning.
A second test would be to verify that a new multi-appserver deployment runs smoothly and deactivates the old servers.
This PR also removes the reconfigure_load_balancers
task, which was requested to be removed by @lgp171188 due to obsolescence.
This is a part 1 fix that will solve the immediate problem of not having enough workers to process the provisioning of servers. A set of follow-up PRs are coming which will handle the implementation of Huey priorities to further clear the technical debt surrounding this problem.