Skip to content

Autoscale bot instances with runtime context

Since the Load Balancer can handle N shards to M bot instances, we should take advantage of autoscaling the bot instances independently. I propose the following strategy:

  • Give every bot instance a full CPU core
  • Collect the following metrics:
    • Events per guild per hour (GuildEvents)
    • Events per instance per hour with their mCPU (InstanceEvents)
  • Average the InstanceEvents to determine the events each instance can handle
  • Every 6 hours, perform the following operation:
    • Using the average events per hour for the whole bot, determine how many instances are needed
    • Rescale the instances to the appropriate amount
    • Remap a distribution of guilds to instances, allowing for multiple instances per guild
    • Use said map to balance out the events across instances