Autoscale bot instances with runtime context
Since the Load Balancer can handle N shards to M bot instances, we should take advantage of autoscaling the bot instances independently. I propose the following strategy:
- Give every bot instance a full CPU core
- Collect the following metrics:
- Events per guild per hour (GuildEvents)
- Events per instance per hour with their mCPU (InstanceEvents)
- Average the InstanceEvents to determine the events each instance can handle
- Every 6 hours, perform the following operation:
- Using the average events per hour for the whole bot, determine how many instances are needed
- Rescale the instances to the appropriate amount
- Remap a distribution of guilds to instances, allowing for multiple instances per guild
- Use said map to balance out the events across instances